Image processing device and image processing method

ABSTRACT

A decision unit decides whether or not a deblocking filter is applied to color difference components of pixels belonging to a line orthogonal to a block boundary in two adjacent blocks adjacent to each other sandwiching the block boundary of the decoded image, by using color difference components of pixels belonging to a line identical to a line used to decide whether or not the deblocking filter is applied to luminance components of the pixels belonging to the line orthogonal to the block boundary. The filtering unit applies the deblocking to the color difference components of the pixels for which it is decided that the deblocking filter is applied. The present technology can be applied to, for example, a case where encoding and decoding of an image are performed.

TECHNICAL FIELD

The present technology relates to an image processing device and animage processing method, and more particularly to an image processingdevice and an image processing method that make it possible to unifyprocesses of a luminance component and a color difference component, forexample.

BACKGROUND ART

In Joint Video Experts Team (JVET) that is a joint standardizationorganization of ITU-T and ISO/IEC, for the purpose of further improvingcoding efficiency compared to H.265/HEVC, standardization work ofVersatile Video Coding (VVC) is underway that is a next-generation imagecoding method.

In the standardization work of VVC, in Non-Patent Document 1, a methodhas been devised that the deblocking filter that can be applied to thecolor difference component is changed to two types similarly to thedeblocking filter that can be applied to the luminance component, andthe strong filter can be applied also to the color difference component.

CITATION LIST Non-Patent Document

-   Non-patent document 1: Jianle Chen, Yan Ye, Seung Hwan Kim:    Algorithm description for Versatile Video Coding and Test Model 2    (VTM 2), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and    ISO/IEC JTC 1/SC 29/WG 11 11th Meeting, Ljubljana, SI, 10-18 Jul.    2018.

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In Non-Patent Document 1, the processes of the luminance component andthe color difference component are not unified.

The present technology has been made in view of such a situation, andmakes it possible to unify the processes of the luminance component andthe color difference component.

Solutions to Problems

A first image processing device of the present technology is an imageprocessing device including: a decoding unit that decodes a bitstream togenerate a decoded image; a decision unit that decides whether or not adeblocking filter is applied to color difference components of pixelsbelonging to a line orthogonal to a block boundary in two adjacentblocks adjacent to each other sandwiching the block boundary of thedecoded image generated by the decoding unit, by using color differencecomponents of pixels belonging to a line identical to a line used todecide whether or not the deblocking filter is applied to luminancecomponents of the pixels belonging to the line orthogonal to the blockboundary; and a filtering unit that applies the deblocking filter to thecolor difference components of the pixels for which it is decided by thedecision unit that the deblocking filter is applied.

A first image processing method of the present technology is an imageprocessing method including: decoding a bitstream to generate a decodedimage; deciding whether or not a deblocking filter is applied to colordifference components of pixels belonging to a line orthogonal to ablock boundary in two adjacent blocks adjacent to each other sandwichingthe block boundary of the decoded image, by using color differencecomponents of pixels belonging to a line identical to a line used todecide whether or not the deblocking filter is applied to luminancecomponents of the pixels belonging to the line orthogonal to the blockboundary; and applying the deblocking filter to the color differencecomponents of the pixels for which it is decided that the deblockingfilter is applied.

In the first image processing device and image processing method of thepresent technology, a bitstream is decoded to generate a decoded image.Furthermore, whether or not a deblocking filter is applied to colordifference components of pixels belonging to a line orthogonal to ablock boundary in two adjacent blocks adjacent to each other sandwichingthe block boundary of the decoded image, is decided by using colordifference components of pixels belonging to a line identical to a lineused to decide whether or not the deblocking filter is applied toluminance components of the pixels belonging to the line orthogonal tothe block boundary. Then, the deblocking filter is applied to the colordifference components of the pixels for which it is decided that thedeblocking filter is applied.

A second image processing device of the present technology is an imageprocessing device including: a decision unit that decides whether or nota deblocking filter is applied to color difference components of pixelsbelonging to a line orthogonal to a block boundary in adjacent blocksadjacent to each other sandwiching the block boundary of a locallydecoded image locally decoded when an image is encoded, by using colordifference components of pixels belonging to a line identical to a lineused to decide whether or not the deblocking filter is applied toluminance components of the pixels belonging to the line orthogonal tothe block boundary; a filtering unit that applies the deblocking filterto the color difference components of the pixels for which it is decidedby the decision unit that the deblocking filter is applied, to generatea filter image; and an encoding unit that encodes the image by using thefilter image generated by the filtering unit.

A second image processing method of the present technology is an imageprocessing method including: deciding whether or not a deblocking filteris applied to color difference components of pixels belonging to a lineorthogonal to a block boundary in adjacent blocks adjacent to each othersandwiching the block boundary of a locally decoded image locallydecoded when an image is encoded, by using color difference componentsof pixels belonging to a line identical to a line used to decide whetheror not the deblocking filter is applied to luminance components of thepixels belonging to the line orthogonal to the block boundary; applyingthe deblocking filter to the color difference components of the pixelsfor which it is decided that the deblocking filter is applied, togenerate a filter image; and encoding the image by using the filterimage.

In the second image processing device and image processing method of thepresent technology, whether or not a deblocking filter is applied tocolor difference components of pixels belonging to a line orthogonal toa block boundary in adjacent blocks adjacent to each other sandwichingthe block boundary of a locally decoded image locally decoded when animage is encoded, is decided by using color difference components ofpixels belonging to a line identical to a line used to decide whether ornot the deblocking filter is applied to luminance components of thepixels belonging to the line orthogonal to the block boundary.Furthermore, the deblocking filter is applied to the color differencecomponents of the pixels for which it is decided that the deblockingfilter is applied, and a filter image is generated. Then, the image isencoded by using the filter image.

Note that, the image processing device can be implemented by causing acomputer to execute a program. The program can be provided by beingrecorded on a recording medium or by being transmitted via atransmission medium.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram explaining a method of calculating bS in HEVC.

FIG. 2 is a diagram explaining a method of calculating bS.

FIG. 3 is an explanatory diagram illustrating an example of colordifference components (U component and V component) in two blocks Bp andBq adjacent to each other sandwiching a vertical block boundary BB.

FIG. 4 is a diagram illustrating an example of a color format (chromaformat) of an image.

FIG. 5 is a diagram explaining filtering decision for a deblockingfilter applied to (pixels in the horizontal direction orthogonal to) avertical block boundary.

FIG. 6 is a diagram explaining filtering decision of a deblocking filterapplied to (pixels in the vertical direction orthogonal to) a horizontalblock boundary.

FIG. 7 is a block diagram illustrating a configuration example of anembodiment of an image processing system to which the present technologyis applied.

FIG. 8 is a block diagram illustrating a detailed configuration exampleof an encoder 11.

FIG. 9 is a flowchart explaining an example of an encoding process bythe encoder 11.

FIG. 10 is a block diagram illustrating a detailed configuration exampleof a decoder 51.

FIG. 11 is a flowchart explaining an example of a decoding process bythe decoder 51.

FIG. 12 is a block diagram illustrating a configuration example of adeblocking filter 31 a.

FIG. 13 is a flowchart explaining a process of the deblocking filter 31a.

FIG. 14 is a diagram explaining filtering decision in a case where thecolor format is the YUV420 format.

FIG. 15 is a diagram explaining filtering decision in a case where thecolor format is the YUV444 format.

FIG. 16 is a diagram explaining filtering decision in a case where thecolor format is the YUV422 format.

FIG. 17 is a diagram explaining filtering decision in a case where thecolor format is the YUV422 format.

FIG. 18 is a block diagram illustrating a configuration example of anembodiment of a computer.

MODE FOR CARRYING OUT THE INVENTION

The scope disclosed in the present specification is not limited to thecontent of the embodiments, and the content of the following referencedocuments REF1 to REF8 known at the time of filing is also incorporatedherein by reference. That is, the content described in the followingreference documents REF1 to REF8 is also a basis for determining supportrequirements. For example, even in a case where a Quad-Tree BlockStructure described in the reference document REF2, a Quad Tree PlusBinary Tree (QTBT) Block Structure described in the reference documentREF3, and a Multi-type Tree (MTT) Block Structure described in thereference documents REF4, REF5, and REF8 are not directly defined in thedetailed description of the invention, they are still within the scopeof the present disclosure and shall meet the support requirements of theclaims. Furthermore, similarly, even in a case where technical terms,for example, parsing, syntax, semantics, and the like are not directlydefined in the detailed description of the invention, they are stillwithin the scope of the present disclosure and shall meet the supportrequirements of the claims.

REF1: Recommendation ITU-T H.264 (April 2017) “Advanced video coding forgeneric audiovisual services”, April 2017

REF2: Recommendation ITU-T H.265 (December 2016) “High efficiency videocoding”, Dec. 2016

REF3: J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, J. Boyce,“Algorithm Description of Joint Exploration Test Model (JEM7)”,JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 andISO/IEC JTC 1/SC 29/WG 11 7th Meeting: Torino, IT, 13-21 Jul. 2017

REF4: B. Bross, J. Chen, S. Liu, “Versatile Video Coding (Draft 3),”JVET-L1001, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 andISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Macau, CN, 312 October 2018

REF5: J. J. Chen, Y. Ye, S. Kim, “Algorithm description for VersatileVideo Coding and Test Model 3 (VTM 3)”, JVET-L1002, Joint Video ExpertsTeam (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12thMeeting: Macau, CN, 312 October 2018

REF6: J. Boyce (Intel), Y. Ye (InterDigital), Y.-W. Huang (Mediatek), M.Karczewicz (Qualcomm), E. Francois (Technicolor), W. Husak (Dolby), J.Ridge (Nokia), A. Abbas (GoPro), “Two tier test model”, JVET-J0093,Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29/WG 11 10th Meeting: San Diego, US, 1020 April 2018

REF7: S. De-Luxan-Hernandez, V. George, J. Ma, T. Nguyen, H. Schwarz, D.Marpe, T. Wiegand (HHI), “CE3: Intra Sub-Partitions Coding Mode (Tests1.1.1 and 1.1.2)”, JVET-M0102, Joint Video Experts Team (JVET) of ITU-TSG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13th Meeting: Marrakech, Mass.,918 January 2019

REF8: M. Ikeda, T. Suzuki (Sony), D. Rusanovskyy, M. Karczewicz(Qualcomm), W. Zhu, K. Misra, P. Cowan, A. Segall (Sharp Labs ofAmerica), K. Andersson, J. Enhorn, Z. Zhang, R. Sjoberg (Ericsson),“CE11.1.6, CE11.1.7 and CE11.1.8: Joint proposals for long deblockingfrom Sony, Qualcomm, Sharp, Ericsson”, JVET-M0471, Joint Video ExpertsTeam (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 13thMeeting: Marrakesh, Mass., 918 January 2019

<Definition>

In this application, the following terms are defined as follows.

Color difference-related parameters mean all parameters related to colordifference. For example, the color difference-related parameters mayinclude information regarding orthogonal transform coefficients of colordifference component, for example, orthogonal transform coefficients(quantization coefficients) of the color difference component includedin any blocks such as a Transform Unit (TU), a Prediction Unit (PU), aCoding Unit (CU), and others, a flag indicating presence or absence of asignificant coefficient (non-zero orthogonal transform coefficient) ofthe color difference component in each block, and the like. The colordifference-related parameters are not limited to such examples, and maybe various parameters related to the color difference.

Application necessity of a deblocking filter means whether or not thedeblocking filter is applied. The application necessity decision of thedeblocking filter means deciding whether or not the deblocking filter isapplied. Furthermore, a decision result of the application necessitydecision is a result of deciding whether or not the deblocking filter isapplied. The decision result of the application necessity decision maybe information indicating either “apply” or “not apply”.

The filtering strength decision means deciding (determining) filteringstrength of a deblocking filter in a case where the deblocking filter isapplied. For example, in a case where there are a weak filter, and achroma long filter having a larger number of taps, that is, strongerfiltering strength, than the weak filter, as a deblocking filter of thecolor difference component, in the filtering strength decision, it isdecided (determined) which of the weak filter and the chroma long filteris used as the deblocking filter to be applied to the color differencecomponent.

Regarding the deblocking filter of the color difference component, theapplication necessity decision and the filtering strength decisiondecide that the deblocking filter is not applied or a type of thedeblocking filter to be applied.

For example, in a case where there are the weak filter and the chromalong filter as the deblocking filter of the color difference component,in the application necessity decision and the filtering strengthdecision, it is decided that the deblocking filter is not applied to thecolor difference component, the weak filter is applied, or the chromalong filter is applied.

Hereinafter, the application necessity decision and the filteringstrength decision are also collectively referred to as filteringdecision.

<Overview of Deblocking Filter>

A process related to a deblocking filter in an existing image codingmethod such as HEVC includes filtering decision (application necessitydecision and filtering strength decision) and filtering (application ofa filter). In the following, an overview of the deblocking filter willbe described using HEVC as an example.

Note that, in the following, the deblocking filter for the colordifference component of the decoded image (including a locally decodedimage locally decoded at the time of encoding) will be described, andthe description of the deblocking filter for the luminance componentwill be omitted as appropriate.

As a process related to the deblocking filter, first, filtering decisionis performed. In the filtering decision, first, application necessitydecision is performed for deciding whether or not the deblocking filteris applied to the block boundary of the decoded image.

Note that, in HEVC, the block boundary is identified on the basis of ablock structure of a Quad-Tree Block Structure described in thereference document REF2. Specifically, among edges of an 8×8 pixel block(sample grid) that is the minimum block unit, an edge that satisfies acondition that the edge is at least one of a Transform Unit (TU)boundary or a Prediction Unit (PU) boundary is identified as the blockboundary in HEVC.

The application necessity decision is performed on the basis of boundarystrength (hereinafter also referred to as bS) of the block boundary. InHEVC, when four lines in a direction orthogonal to a partial blockboundary (a part of the block boundary), which is a unit of processingwhen the filtering decision (application necessity decision) of thedeblocking filter is performed on the block boundary, are defined as aunit of filter application to which the deblocking filter is applied,the bS is calculated every four lines that are the unit of filterapplication. In a case where the block boundary is a vertical boundary,a line of the unit of filter application is a line (row) in thehorizontal direction orthogonal to the vertical boundary.

Furthermore, in a case where the block boundary is a horizontalboundary, a line of the unit of filter application is a line (column) inthe vertical direction orthogonal to the horizontal boundary.

FIG. 1 is a diagram explaining a method of calculating bS in HEVC.

As illustrated in FIG. 1, in HEVC, the bS is calculated on the basis ofthe truth or falsehood (satisfied or not satisfied) of a condition Athat is a condition related to intra prediction, a condition B1 that isa condition related to a significant coefficient of the Y component, anda condition B2 that is a condition related to a motion vector (MV) and areference picture. Referring to FIG. 1, the bS is set to 2 in a casewhere the condition A is true. Furthermore, in a case where thecondition A is false and at least one of the condition B1 or thecondition B2 is true, the bS is set to 1. Then, in a case where thecondition A, the condition B1, and the condition B2 are all false, thebS is set to 0. Note that, the conditions A, B1, and B2 illustrated inFIG. 1 are as follows. Furthermore, here, for the sake of simplicity,the block boundary is assumed to be a vertical boundary.

Condition A: Among Coding Units (CUs) including pixels of the uppermostline among lines orthogonal to the block boundary that is a calculationtarget of the bS and sandwiching the block boundary, an encoding mode ofat least one of the CUs is an intra prediction mode.

Condition B1: The block boundary is the TU boundary, and among two TUsincluding pixels of the uppermost line among lines orthogonal to theblock boundary that is a calculation target of the bS and sandwichingthe block boundary, the significant coefficient of the Y componentexists in at least one of the TUs.

Condition B2: Between two CUs including pixels of the uppermost lineamong lines orthogonal to the block boundary that is a calculationtarget of the bS and sandwiching the block boundary, an absolute valueof a difference between MVs is one pixel or more, or reference picturesof motion compensation are different from each other or the numbers ofMVs are different from each other.

Moreover, in HEVC, a deblocking filter for the luminance component (Ycomponent) of the decoded image can be applied to the block boundary forwhich the bS set as described above is set to greater than or equalto 1. For that reason, in HEVC, the decision result of the applicationnecessity decision of the deblocking filter for the luminance componentof the decoded image may differ depending on whether or not thecondition B1 and the condition B2 are satisfied.

Note that, in HEVC, as the deblocking filter for the luminance componentof the decoded image, the strong filter having a high filtering strengthand the weak filter having a low filtering strength are prepared. In acase where the bS is greater than or equal to 1, in a process related tothe deblocking filter for the luminance component of the decoded image,application necessity decision based on additional conditions is furtherperformed, and then decision of the filtering strength and filtering areperformed. Details of these processes are described in the referencedocument REF2, and the description thereof is omitted here.

On the other hand, a deblocking filter for the color differencecomponents (U component, V component) of the decoded image in HEVC isapplied only to the block boundary whose bS is 2. For that reason, asillustrated in FIG. 1, whether or not the conditions B1 and B2 aresatisfied does not affect the application necessity decision of thedeblocking filter to the color difference components of the decodedimage, in HEVC.

Furthermore, in HEVC, the deblocking filter that can be applied to thecolor difference components of the decoded image is only the weakfilter. For that reason, a filtering strength decision process is notnecessary for the color difference components of the decoded image, andin a case where the bS is 2, the weak filter is applied to the colordifference components of the decoded image.

By the way, as described in the reference document REF3, in the blockdivision by the QTBT Block Structure in VVC, a block having a largersize can be selected than that in the block division by the Quad-TreeBlock Structure in HEVC. In a case where the block size is large in aflat area (an area in which a change in pixel value in the area issmall), block distortion is likely to occur. For that reason, in VVC inwhich a block having a larger size can be selected, in a case where thedeblocking filter that can be applied to the color difference componentsof the decoded image is only the weak filter similarly to HEVC, therehas been a possibility that a remarkable block distortion remains in thecolor difference components. In view of such a situation, it is desiredto improve the deblocking filter for the color difference components ofthe decoded image.

Thus, in Non-Patent Document 1, a method of applying the deblockingfilter has been devised, which is different from that in HEVC. In themethod of applying the deblocking filter of Non-Patent Document 1, ithas been devised that, for example, the deblocking filter that can beapplied to the color difference component is changed to two typessimilarly to the deblocking filter that can be applied to the luminancecomponent, and the strong filter can be applied also to the colordifference component. Furthermore, it has also been devised that thedeblocking filter can be applied to the color difference components ofthe decoded image not only in a case where the bS is 2 but also in acase where the bS is 1.

FIG. 2 is a diagram explaining a method of calculating bS in the methodof applying the deblocking filter of Non-Patent Document 1.

In the method of applying the deblocking filter of Non-Patent Document1, the bS is calculated on the basis of the conditions A, B1, and B2described above, similarly to the example in HEVC illustrated in FIG. 2.However, as described above, the deblocking filter can be applied to thecolor difference components of the decoded image not only in the casewhere the bS is 2 but also in the case where the bS is 1. For thatreason, as illustrated in FIG. 2, the decision result of the applicationnecessity decision of the deblocking filter for the color differencecomponents (U component, V component) of the decoded image may differdepending on whether or not the condition B1 and the condition B2 aresatisfied.

Hereinafter, a description will be given of filtering decision(application necessity decision and filtering strength decision) andfiltering regarding a deblocking filter that can be applied to the colordifference component of the decoded image in Non-Patent Document 1.

FIG. 3 is a diagram illustrating an example of pixels of colordifference components (U component and V component) of a block Bp and ablock Bq as two adjacent blocks adjacent to each other sandwiching avertical block boundary BB that is a block boundary in the verticaldirection.

Note that, here, the vertical block boundary will be described as anexample, but the matters described for the vertical block boundary canbe similarly applied to the horizontal block boundary that is the blockboundary in the horizontal direction unless otherwise specified.Furthermore, although FIG. 3 illustrates an example in which the blockBp and the block Bq of the color difference component are blocks of 4×4pixels, the matters described here can be similarly applied to blocks ofother sizes.

In the example of FIG. 3, the color difference component (and pixels ofthe color difference component) in the block Bp are indicated by symbolsp_(i, j). The i is a column index and the j is a row index. The columnindexes i are numbered 0, 1, 2, and 3 in order from a column closest tothe vertical block boundary BB (from left to right in the figure). Therow indexes j are numbered 0, 1, 2, and 3 from top to bottom. On theother hand, the color difference component (and pixels of the colordifference component) in the block Bq are indicated by symbols q_(k j).The k is a column index and the j is a row index. The column indexes kare numbered 0, 1, 2, and 3 in order from a column closest to thevertical block boundary BB (from right to left in the figure).

Note that, here, the block boundary BB is assumed to be the verticalblock boundary, but the block boundary BB can be regarded as thehorizontal block boundary, and the block Bp and the block Bq can beregarded as two adjacent blocks adjacent to each other sandwiching thehorizontal block boundary BB. In this case, in the p_(i, j), the i is arow index and the j is a column index. The same applies to the q_(k j).

After the bS is calculated as described with reference to FIG. 2, thefiltering decision is performed using the following three conditions. Ina case where the color format of the decoded image is, for example, theYUV420 format, the filtering decision is performed every two lines ofthe color difference components.

That is, in the case where the color format of the decoded image is theYUV420 format, a partial vertical block boundary that is a unit ofprocessing when it is decided whether or not the deblocking filter isapplied to (pixels in the horizontal direction orthogonal to) thevertical block boundary BB, is a vertical block boundary for two linesof the color difference components continuous in the vertical direction,and orthogonal to two lines of the color difference components.

The filtering decision for the vertical block boundary BB is performedfor each partial vertical block boundary.

In the example illustrated in FIG. 3, the filtering decision isperformed separately for a partial vertical block boundary b1 for twolines of the line L11 and the line L12, and a partial vertical blockboundary b2 for two lines of the line L21 and the line L22.

The filtering decision for the partial vertical block boundary b1 isperformed using the line L11 and the line L12 (of the color differencecomponents) in the horizontal direction orthogonal to the partialvertical block boundary b1. Similarly, the filtering decision for thepartial vertical block boundary b2 is performed using the line L21 andthe line L22 in the horizontal direction orthogonal to the partialvertical block boundary b2.

In the following, a description will be given of filtering decision andfiltering performed for the partial vertical block boundary b1.

In the filtering decision, in the adaptation necessity decision, it isdecided in order whether or not a condition C91 and a condition C92below are true.

(bS==2∥bS ==1 && (block width>16 && block_height>16))   Condition C91

d<beta   Condition C92

Note that, in the condition C91, the block_width and the block_heightare the horizontal size and the vertical size of a block (for example,CU) over the partial vertical block boundary b1 to be subjected to thefiltering decision, as illustrated in FIG. 3. ∥ represents a logical sumoperation, and && represents a logical product operation.

Furthermore, the variable beta in the condition C92 is an edge decisionthreshold value, and the variable beta is given depending on aquantization parameter. Furthermore, the variable d in the condition C92is calculated by the following equations (1) to (7).

dp0=Abs(p _(2, 0)−2*p _(1, 0) +p _(0, 0))   (1)

dp1=Abs(p _(2, 1)−2*p _(1, 1) +p _(0, 1))   (2)

dq0=Abs(q _(2, 0)−2*q _(1, 0) +q _(0, 0))   (3)

dq1=Abs(q _(2, 1)−2*q _(1, 1) +q _(0, 1))   (4)

dpq0=dp0+dq0   (5)

dpq1=dp1+dq1   (6)

d=dpq0+dpq1   (7)

Note that, the condition C92 is similar to a condition used in thefiltering decision of the deblocking filter applied to the luminancecomponent in HEVC (hereinafter, referred to as a condition in theluminance component) except that a line referred to is different. In thecondition of the luminance component, pixels of the first line andpixels of the fourth line are referred to, and the decision is performedevery four lines (segments). In the YUV420 format, the pixel density ineach of the horizontal direction and vertical direction of the colordifference components (U component and V component) are half the pixeldensity of the luminance component, so that four lines of the luminancecomponents correspond to two lines of the color difference components.Regarding the condition C92, pixels of the two lines L11 and L12 of thecolor difference components corresponding to the four lines of theluminance components are referred to, and the decision is performedevery two lines.

In a case where at least one of the condition C91 or the condition C92is false, the deblocking filter is not applied to the color differencecomponents of the decoded image. On the other hand, in a case where boththe condition C91 and the condition C92 are true, the filtering strengthdecision is performed in the filtering decision.

In the filtering strength decision, as a decision of which of the strongfilter and the weak filter is applied, it is decided whether or not acondition C93 below is true.

Condition C93: (block_width>16 && block_height>16)

Note that, the block_width and block height in the condition C93 are thehorizontal size and the vertical size of a block over the partialvertical block boundary b1 to be subjected to the filtering decision,similarly to the block_width and block_height in the condition C91.

In a case where the condition C93 is true, the strong filter is appliedto the color difference components of the decoded image at the partialvertical block boundary b1, and in a case where the condition C93 isfalse, the weak filter is applied to the color difference components ofthe decoded image at the partial vertical block boundary b1.

The strong filter applied to the color difference component inNon-Patent Document 1 is similar to the strong filter applied to theluminance component in HEVC, and is represented by the followingequations (8) to (13).

p ₀′=Clip3(p ₀−2*tc, p ₀+2*t _(c), (p ₂+2*p ₁+2*p ₂+2*q ₀ +q ₁+4)>>3)  (8)

p ₁′=Clip3(p ₁−2*tc, p ₁+2*t _(c), (p ₂ +p ₁ +p ₀ +q ₂+2)>>2)   (9)

p ₂′=Clip3(p ₂−2*tc, p ₂+2*t _(c), (2*p ₃+3* p ₂ +p ₁ +p ₀ +q ₀+4)>>3)  (10)

p ₀′=Clip3(q ₀−2*tc, q ₀+2*t _(c), (p ₁+2p ₀+2q ₀+2q ₁ +q ₂+4)>>3)  (11)

q ₁′=Clip3(q ₁−2*tc, q ₁+2*t _(c), (p ₀ +q ₀ +q ₁ +q ₂+2)>>2)   (12)

q ₂′=Clip3(q ₂−2*t _(c) , q ₂+2*t _(c), (p ₀ +q ₀ +q ₁+3*q ₂+2*q₃+4)>>3)   (13)

Note that, in the equations (8) to (13), p_(i) and q_(k) are pixelvalues (color difference components) of the pixels of the colordifference components (hereinafter, also referred to as color differencepixels) before the application of the deblocking filter. Furthermore,p_(i)′ and q_(k)′ are the color difference components of the colordifference pixels after the deblocking filter is applied. Here, the iand k are column indexes in the block Bp and the block Bq describedabove, respectively, and row indexes are omitted since they are the samein equations (8) to (13). Furthermore, the t_(c) is a parameter givendepending on the quantization parameter. Furthermore, the Clip3(a, b, c)represents a clipping process in which the value c is clipped in a rangeofa□c□b.

The weak filter applied to the color difference components in Non-PatentDocument 1 is the same as the weak filter applied to the colordifference components in HEVC.

In the above, the process has been described related to the deblockingfilter that can be applied to the color difference components of thedecoded image in Non-Patent Document 1. According to the methoddescribed above, the strong filter is applied not only to the luminancecomponent but also to the color difference components depending on theconditions.

<Color Format>

FIG. 4 is a diagram illustrating an example of a color format (chromaformat) of an image.

Examples of the color format of the image to be encoded include theYUV420 format, the YUV422 format, the YUV444 format, and the like. Notethat, the color format of the image to be encoded is not limited tothese.

In the YUV420 format, the densities in the horizontal direction and thevertical direction of (the pixels of) the color difference components(chroma) are down-sampled to ½ of the densities in the horizontaldirection and the vertical direction of (the pixels of) the luminancecomponent (luminance), respectively. In the YUV422 format, the densityin the vertical direction of the color difference component is the sameas the density in the vertical direction of the luminance component, butthe density in the horizontal direction of the color differencecomponent is down-sampled to ½ of the density in the horizontaldirection of the luminance component. In the YUV444 format, thedensities in the horizontal direction and the vertical direction of thecolor difference components are the same as the densities in thehorizontal direction and the vertical direction of the luminancecomponent, respectively.

Note that, in FIG. 5, the arrows indicate the scanning order of theluminance component and the color difference component.

In the reference document REF4, regarding images in the YUV420 format,the YUV422 format, and the YUV444 format, it has been devised to performthe filtering decision using two lines, the first line and the fourthline, of the four lines (segments) for the luminance component, andperform the filtering decision using two lines, the first line and thesecond line, of the two lines (segments) or the four lines (segments)for the color difference component.

Here, since the density of the color difference component in thevertical direction in the YUV422 format is the same as the density ofthe luminance component in the vertical direction, the filteringdecision for the color difference component at the vertical blockboundary (block boundary in the vertical direction) is performed inunits of four lines similarly to the filtering decision for theluminance component. The same applies to the color difference componentin the horizontal direction and the vertical direction in the YUV444format.

On the other hand, since the density of the color difference componentin the horizontal direction in the YUV422 format is ½ of the density ofthe luminance component in the horizontal direction, the filteringdecision for the color difference component at the horizontal blockboundary (block boundary in the horizontal direction) is performed inunits of two lines that is ½ of the units of four lines of the filteringdecision. The same applies to the color difference component in thehorizontal direction and the vertical direction in the YUV420 format.

Thus, in the reference document REF4, for the YUV420 format, althoughthe densities of the color difference component in the horizontaldirection and the vertical direction are ½ of the densities of theluminance component in the horizontal direction and the verticaldirection, respectively, the filtering decision for the color differencecomponent in the vertical direction is performed using two linessimilarly to the filtering decision for the luminance component in thehorizontal direction and the vertical direction. For this reason, thereis a possibility that a difference occurs in accuracy between thefiltering decision for the color difference component and the filteringdecision for the luminance component, and the image quality degrades.The same applies to the color difference component and the luminancecomponent in the horizontal direction in the YUV422 format.

To make the accuracy of the filtering decision for the color differencecomponent and the filtering decision for the luminance component aboutthe same, for the horizontal direction and the vertical direction in theYUV420 format and the horizontal direction in the YUV422 format, it isdesirable to use one line out of the two lines for the filteringdecision for the color difference component in correspondence with theuse of the two lines out of the four lines for the filtering decisionfor the luminance component.

Furthermore, although the density of the color difference component inthe vertical direction in the YUV422 format is the same as the densityof the luminance component in the vertical direction, the filteringdecision for the luminance component in the vertical direction isperformed using two lines, the first line and the fourth line, of thefour lines, whereas the filtering decision for the color differencecomponent in the vertical direction is performed using the two linesdifferent from the case of the luminance component of the four lines,that is, the two lines, the first line and the second line. For thisreason, there is a possibility that a difference occurs in accuracybetween the filtering decision for the color difference component andthe filtering decision for the luminance component, and the imagequality degrades. The same applies to the color difference component andthe luminance component in the horizontal direction and the verticaldirection in the YUV444 format.

To make the accuracy of the filtering decision for the color differencecomponent and the filtering decision for the luminance component aboutthe same, for the vertical direction in the YUV422 format and thehorizontal direction and the vertical direction of the YUV444 format, itis desirable to use the first line and the fourth line out of the fourlines for the filtering decision for the color difference component incorrespondence with the use of the first line and the fourth line out ofthe four lines for the filtering decision for the luminance component.

Thus, in the present technology, the number of reference lines to bereferred to in Deblocking filter decision is changed depending on thecolor format (YUV420/422/444). That is, in the present technology, thenumber of reference lines used for the filtering decision of the colordifference component is set depending on the color format.

For example, in a case where the color format is the YUV420 format,down-sampling is performed in the horizontal direction and the verticaldirection, so that one line is set as the reference line for both thehorizontal and vertical block boundaries. For example, in a case wherethe color format is YUV422 format, down-sampling is performed in thehorizontal direction, so that one line is set as the reference line atthe block boundary in the horizontal direction, and two lines (Luma(luminance component) are set as the reference lines at the blockboundary in the vertical direction. For example, in a case where thecolor format is YUV444 format, two lines (same as Luma) are set as thereference lines for both the horizontal and vertical block boundaries.

Moreover, in the present technology, by matching the line (referenceline) used for the filtering decision for the color difference componentwith the line used for the filtering decision for the luminancecomponent, correspondence is made between the line used for thefiltering decision for the color difference component and the line usedfor the filtering decision for the luminance component, whereby theprocesses of the luminance component and the color difference componentare unified.

FIGS. 5 and 6 are diagrams explaining an overview of the presenttechnology.

FIG. 5 is a diagram explaining filtering decision (hereinafter, alsoreferred to as vertical block boundary filtering decision) for adeblocking filter applied to (pixels in the horizontal directionorthogonal to) a vertical block boundary.

A of FIG. 5 illustrates luminance components and color differencecomponents in the YUV420 format. B of FIG. 5B illustrates luminancecomponents and color difference components in the YUV422 format. C ofFIG. 5 illustrates luminance components and color difference componentsin the YUV444 format.

In the present technology, for the luminance component, for example, inany color format of the YUV420 format, the YUV422 format, and the YUV444format, as devised in the reference document REF4, with a vertical blockboundary for four lines in the horizontal direction as a partialvertical block boundary, the vertical block boundary filtering decisionis performed using two lines, a first line D1 and a fourth line D4, ofthe four lines in the horizontal direction orthogonal to the partialvertical block boundary, for each partial vertical block boundary.

Furthermore, in the present technology, for the color differencecomponent in the YUV420 format, the density in the vertical direction is½ of that of the luminance component, so that the vertical blockboundary filtering decision is performed using only a first line D11 outof two lines in the horizontal direction of the color differencecomponent corresponding to the four lines in the horizontal direction ofthe luminance component.

Moreover, in the present technology, for the color difference componentof the YUV422 format or the YUV444 format, the density in the verticaldirection is the same as that of the luminance component, so that thevertical block boundary filtering decision is performed using two lines,a first line D21 and a fourth line D24, or a first line D31 and a fourthline D34, of the four lines in the horizontal direction of the colordifference component corresponding to the four lines in the horizontaldirection of the color difference component, similarly to the luminancecomponent.

FIG. 6 is a diagram explaining filtering decision (hereinafter, alsoreferred to as horizontal block boundary filtering decision) for adeblocking filter applied to (pixels in the vertical directionorthogonal to) a horizontal block boundary.

A of FIG. 6 illustrates luminance components and color differencecomponents in the YUV420 format. B of FIG. 6 illustrates luminancecomponents and color difference components in the YUV422 format. C ofFIG. 6 illustrates luminance components and color difference componentsin the YUV444 format.

In the present technology, for the luminance component, in any colorformat of the YUV420 format, the YUV422 format, and the YUV444 format,as devised in the reference document REF4, with a horizontal blockboundary for four lines in the vertical direction as a partialhorizontal block boundary, the horizontal block boundary filteringdecision is performed using two lines, a first line D51 and a fourthline D54, of the four lines in the vertical direction orthogonal to thepartial horizontal block boundary, for each partial horizontal blockboundary.

The partial horizontal block boundary is a unit of processing when it isdecided whether or not the deblocking filter is applied (the pixels inthe vertical direction orthogonal to) the horizontal block boundary,similarly to the partial vertical block boundary.

Furthermore, in the present technology, for the color differencecomponent in the YUV420 format or the YUV422, the density in thehorizontal direction is ½ of that of the luminance component, so thatthe horizontal block boundary filtering decision is performed using onlya first line D61 or D71 of the two lines in the vertical direction ofthe color difference component corresponding to the four lines in thevertical direction of the luminance component.

Moreover, in the present technology, for the color difference componentin the YUV444 format, the density in the horizontal direction is thesame as that of the luminance component, so that the horizontal blockboundary filtering decision is performed using two lines, a first lineD81 and a fourth line D84, of the four lines in the vertical directionof the color difference component corresponding to the four lines in thevertical direction of the luminance component, similarly to theluminance component.

<Image Processing System to which the Present Technology is Applied>

FIG. 7 is a block diagram illustrating a configuration example of anembodiment of an image processing system to which the present technologyis applied.

An image processing system 10 includes an image processing device as anencoder 11, and an image processing device as a decoder 51.

The encoder 11 encodes an original image to be encoded supplied to theencoder 11 and outputs an encoded bitstream obtained by the encoding.The encoded bitstream is supplied to the decoder 51 via a recordingmedium or a transmission medium (not illustrated).

The decoder 51 decodes the encoded bitstream supplied to the decoder 51and outputs a decoded image obtained by the decoding.

<Configuration Example of Encoder 11>

FIG. 8 is a block diagram illustrating a detailed configuration exampleof the encoder 11 of FIG. 7.

Note that, in the block diagram described below, lines for supplyinginformation (data) necessary for a process for each block are omitted asappropriate to avoid complicating the drawing.

In FIG. 8, the encoder 11 includes an A/D conversion unit 21, a screenrearrangement buffer 22, a calculation unit 23, an orthogonal transformunit 24, a quantization unit 25, a lossless encoding unit 26, and anaccumulation buffer 27. Moreover, the encoder 11 includes an inversequantization unit 28, an inverse orthogonal transform unit 29, acalculation unit 30, a frame memory 32, a selection unit 33, an intraprediction unit 34, a motion prediction/compensation unit 35, apredicted image selection unit 36, and a rate control unit 37.Furthermore, the encoder 11 includes a deblocking filter 31 a, anadaptive offset filter 41, and an adaptive loop filter (ALF) 42.

The A/D conversion unit 21 performs A/D conversion of an original imageof an analog signal (encoding target) into an original image of adigital signal, and supplies the original image to the screenrearrangement buffer 22 for storage. Note that, in a case where theoriginal image of the digital signal is supplied to the encoder 11, theencoder 11 can be configured without being provided with the A/Dconversion unit 21.

The screen rearrangement buffer 22 rearranges frames of the originalimage into the encoding (decoding) order from the display orderdepending on a Group Of Picture (GOP), and supplies the frames to thecalculation unit 23, the intra prediction unit 34, and the motionprediction/compensation unit 35.

The calculation unit 23 subtracts a predicted image supplied from theintra prediction unit 34 or the motion prediction/compensation unit 35via the predicted image selection unit 36 from the original image fromthe screen rearrangement buffer 22, and supplies a residual (predictionresidual) obtained by the subtraction to the orthogonal transform unit24.

The orthogonal transform unit 24 performs an orthogonal transform suchas a discrete cosine transform or a Karhunen-Loeve transform on theresidual supplied from the calculation unit 23, and supplies orthogonaltransform coefficients obtained by the orthogonal exchange to thequantization unit 25.

The quantization unit 25 quantizes the orthogonal transform coefficientssupplied from the orthogonal transform unit 24. The quantization unit 25sets a quantization parameter on the basis of a target value of the codeamount (code amount target value) supplied from the rate control unit37, and quantizes the orthogonal transform coefficients. Thequantization unit 25 supplies coded data that is the quantizedorthogonal transform coefficients to the lossless encoding unit 26.

The lossless encoding unit 26 encodes the quantized orthogonal transformcoefficients as coded data from the quantization unit 25 with apredetermined lossless encoding method.

Furthermore, the lossless encoding unit 26 acquires, from each block,encoding information necessary for decoding by the decoding device 170out of encoding information regarding predictive encoding by the encoder11.

Here, the encoding information includes, for example, a prediction modeof intra prediction or inter-prediction, motion information such as amotion vector, the code amount target value, the quantization parameter,a picture type (I, P, B), filter parameters of the deblocking filter 31a and the adaptive offset filter 41, and the like.

The prediction mode can be acquired from the intra prediction unit 34 orthe motion prediction/compensation unit 35. The motion information canbe acquired from the motion prediction/compensation unit 35. The filterparameters of the deblocking filter 31 a and the adaptive offset filter41 can be acquired from the deblocking filter 31 a and the adaptiveoffset filter 41, respectively.

The lossless encoding unit 26 encodes the encoding information with alossless encoding method, for example, variable-length encoding orarithmetic encoding such as Context-Adaptive Variable Length Coding(CAVLC) or Context-Adaptive Binary Arithmetic Coding (CABAC), or others,generates a (multiplexed) encoded bitstream including the encodinginformation after encoding, and the coded data from the quantizationunit 25, and supplies the encoded bitstream to the accumulation buffer27.

The accumulation buffer 27 temporarily stores the encoded bitstreamsupplied from the lossless encoding unit 26. The encoded bitstreamaccumulated in the accumulation buffer 27 is read and transmitted at apredetermined timing.

The coded data that is the orthogonal transform coefficients quantizedby the quantization unit 25 is supplied to the lossless encoding unit 26and also to the inverse quantization unit 28. The inverse quantizationunit 28 performs inverse quantization on the quantized orthogonaltransform coefficients with a method corresponding to the quantizationby the quantization unit 25, and supplies the orthogonal transformcoefficients obtained by the inverse quantization to the inverseorthogonal transform unit 29.

The inverse orthogonal transform unit 29 performs inverse orthogonaltransform on the orthogonal transform coefficients supplied from theinverse quantization unit 28 with a method corresponding to anorthogonal transform process performed by the orthogonal transform unit24, and supplies a residual obtained as a result of the inverseorthogonal transform to the calculation unit 30.

The calculation unit 30 adds the predicted image supplied from the intraprediction unit 34 or the motion prediction/compensation unit 35 via thepredicted image selection unit 36 to the residual supplied from theinverse orthogonal transform unit 29, and therefore obtains and outputs(a part of) a decoded image obtained by decoding the original image.

The decoded image output by the calculation unit 30 is supplied to thedeblocking filter 31 a or the frame memory 32.

The frame memory 32 temporarily stores the decoded image supplied fromthe calculation unit 30, and a decoded image (filter image) to which thedeblocking filter 31 a, the adaptive offset filter 41, and the ALF 42are applied, supplied from the ALF 42. The decoded image stored in theframe memory 32 is supplied to the selection unit 33 at a necessarytiming, as a reference image used for generating the predicted image.

The selection unit 33 selects a supply destination of the referenceimage supplied from the frame memory 32.

In a case where the intra prediction is performed in the intraprediction unit 34, the selection unit 33 supplies the reference imagesupplied from the frame memory 32 to the intra prediction unit 34. In acase where inter-prediction is performed in the motionprediction/compensation unit 35, the selection unit 33 supplies thereference image supplied from the frame memory 32 to the motionprediction/compensation unit 35.

The intra prediction unit 34 performs intra prediction (in-screenprediction) using the original image supplied from the screenrearrangement buffer 22 and the reference image supplied from the framememory 32 via the selection unit 33. The intra prediction unit 34selects an optimal intra prediction mode on the basis of a predeterminedcost function (for example, RD cost, or the like), and supplies apredicted image generated from the reference image in the optimal intraprediction mode to the predicted image selection unit 36. Furthermore,as described above, the intra prediction unit 34 appropriately suppliesthe prediction mode indicating the intra prediction mode selected on thebasis of the cost function to the lossless encoding unit 26 and thelike. [0122]

The motion prediction/compensation unit 35 performs motion prediction(inter-prediction) using the original image supplied from the screenrearrangement buffer 22, and the reference image supplied from the framememory 32 via the selection unit 33. Moreover, the motionprediction/compensation unit 35 performs motion compensation dependingon the motion vector detected by the motion prediction, to generate thepredicted image. The motion prediction/compensation unit 35 performsinter-prediction in a plurality of inter-prediction modes prepared inadvance, to generate a predicted image from the reference image.

The motion prediction/compensation unit 35 selects an optimalinter-prediction mode on the basis of a predetermined cost function ofthe predicted image obtained for each of the plurality ofinter-prediction modes. Moreover, the motion prediction/compensationunit 35 supplies the predicted image generated in the optimalinter-prediction mode to the predicted image selection unit 36.

Furthermore, the motion prediction/compensation unit 35 supplies, to thelossless encoding unit 26, a prediction mode indicating theinter-prediction mode selected on the basis of the cost function, andmotion information such as a motion vector required in decoding of thecoded data encoded in the inter-prediction mode, and the like.

The predicted image selection unit 36 selects a supply source of thepredicted image to be supplied to the calculation units 23 and 30 fromthe intra prediction unit 34 and the motion prediction/compensation unit35, and supplies the predicted image supplied from the selected supplysource to the calculation units 23 and 30.

The rate control unit 37 controls a rate of quantization operation inthe quantization unit 25 on the basis of the code amount of the encodedbitstream accumulated in the accumulation buffer 27 so that overflow orunderflow does not occur. That is, the rate control unit 37 sets atarget code amount of the encoded bitstream not to cause overflow andunderflow of the accumulation buffer 27, and supplies the target codeamount to the quantization unit 25.

The deblocking filter 31 a applies the deblocking filter to the decodedimage from the calculation unit 30 as necessary, and supplies, to theadaptive offset filter 41, the decoded image (filter image) to which thedeblocking filter is applied, or the decoded image to which thedeblocking filter is not applied.

The adaptive offset filter 41 applies the adaptive offset filter to thedecoded image from the deblocking filter 31 a as necessary, andsupplies, to the ALF 42, the decoded image (filter image) to which theadaptive offset filter is applied, or the decoded image to which theadaptive offset filter is not applied.

The ALF 42 applies ALF to the decoded image from the adaptive offsetfilter 41 as necessary, and supplies, to the frame memory 32, thedecoded image to which the ALF is applied or the decoded image to whichthe ALF is not applied.

<Encoding Process>

FIG. 9 is a flowchart explaining an example of an encoding process bythe encoder 11 in FIG. 8.

Note that, the order of the steps of the encoding process illustrated inFIG. 9 is an order for convenience of description, and the steps of theactual encoding process are appropriately performed in parallel and in anecessary order. The same applies to processes described later.

In the encoder 11, in step S11, the A/D conversion unit 21 performs A/Dconversion on the original image and supplies the converted originalimage to the screen rearrangement buffer 22, and the process proceeds tostep S12.

In step S12, the screen rearrangement buffer 22 stores the originalimage from the A/D conversion unit 21 and performs rearrangement in theencoding order to output the original image, and the process proceeds tostep S13.

In step S13, the intra prediction unit 34 performs an intra predictionprocess in the intra prediction mode, and the process proceeds to stepS14. In step S14, the motion prediction/compensation unit 35 performsinter-motion prediction process of performing motion prediction andmotion compensation in the inter-prediction mode, and the processproceeds to step S15.

In the intra prediction process by the intra prediction unit 34 and theinter-motion prediction process by the motion prediction/compensationunit 35, cost functions of various prediction modes are calculated, anda predicted image is generated.

In step S15, the predicted image selection unit 36 determines an optimalprediction mode on the basis of each cost function obtained by the intraprediction unit 34 and the motion prediction/compensation unit 35. Then,the predicted image selection unit 36 selects and outputs a predictedimage in the optimal prediction mode from the predicted image generatedby the intra prediction unit 34 and the predicted image generated by themotion prediction/compensation unit 35, and the process proceeds fromstep S15 to step S16.

In step S16, the calculation unit 23 calculates a residual between atarget image to be encoded that is the original image output from thescreen rearrangement buffer 22, and the predicted image output from thepredicted image selection unit 36, and supplies the residual to theorthogonal transform unit 24, and the process proceeds to step S17.

In step S17, the orthogonal transform unit 24 performs orthogonaltransform on the residual from the calculation unit 23, and suppliesorthogonal transform coefficients obtained as a result of the orthogonaltransform, to the quantization unit 25, and the process proceeds to stepS18.

In step S18, the quantization unit 25 quantizes the orthogonal transformcoefficients from the orthogonal transform unit 24, and suppliesquantization coefficients obtained by the quantization to the losslessencoding unit 26 and the inverse quantization unit 28, and the processproceeds to step S19.

In step S19, the inverse quantization unit 28 performs inversequantization on the quantization coefficients from the quantization unit25, and supplies orthogonal transform coefficients obtained as a resultof the inverse quantization, to the inverse orthogonal transform unit29, and the process proceeds to step S20. In step S20, the inverseorthogonal transform unit 29 performs inverse orthogonal transform onthe orthogonal transform coefficients from the inverse quantization unit28, and supplies a residual obtained as a result of the inverseorthogonal transform, to the calculation unit 30, and the processproceeds to step S21.

In step S21, the calculation unit 30 adds the residual from the inverseorthogonal transform unit 29 and the predicted image output from thepredicted image selection unit 36 together, to generate a decoded imagecorresponding to the original image subjected to residual calculation inthe calculation unit 23. The calculation unit 30 supplies the decodedimage to the deblocking filter 31 a, and the process proceeds from stepS21 to step S22.

In step S22, the deblocking filter 31 a applies the deblocking filter tothe decoded image from the calculation unit 30, supplies a filter imageobtained as a result of the application, to the adaptive offset filter41, and the process proceeds to step S23.

In step S23, the adaptive offset filter 41 applies the adaptive offsetfilter to the filter image from the deblocking filter 31 a, supplies afilter image obtained as a result of the application, to the ALF 42, andthe process proceeds to step S24.

In step S24, the ALF 42 applies the ALF to the filter image from theadaptive offset filter 41, supplies a filter image obtained as a resultof the application, to the frame memory 32, and the process proceeds tostep S25.

In step S25, the frame memory 32 stores the filter image supplied fromthe ALF 42, and the process proceeds to step S26. The filter imagestored in the frame memory 32 is used as a reference image that is asource for generating the predicted image, in steps S13 and S14.

In step S26, the lossless encoding unit 26 encodes the coded data thatis the quantization coefficients from the quantization unit 25, andgenerates an encoded bitstream including the coded data. Moreover, thelossless encoding unit 26 encodes encoding information as necessary,such as the quantization parameter used for quantization in thequantization unit 25, the prediction mode obtained in the intraprediction process in the intra prediction unit 34, the prediction modeand motion information obtained in the inter-motion prediction processin the motion prediction/compensation unit 35, and the filter parametersof the deblocking filter 31 a and the adaptive offset filter 41, andincludes the encoding information in the encoded bitstream.

Then, the lossless encoding unit 26 supplies the encoded bitstream tothe accumulation buffer 27, and the process proceeds from step S26 tostep S27.

In step S27, the accumulation buffer 27 accumulates the encodedbitstream from the lossless encoding unit 26, and the process proceedsto step S28. The encoded bitstream accumulated in the accumulationbuffer 27 is appropriately read and transmitted.

In step S28, the rate control unit 37 controls the rate of thequantization operation in the quantization unit 25 on the basis of thecode amount (generated code amount) of the encoded bitstream accumulatedin the accumulation buffer 27 so that overflow or underflow does notoccur, and the encoding process ends.

<Configuration Example of Decoder 51>

FIG. 10 is a block diagram illustrating a detailed configuration exampleof the decoder 51 of FIG. 7.

In FIG. 10, the decoder 51 includes an accumulation buffer 61, alossless decoding unit 62, an inverse quantization unit 63, an inverseorthogonal transform unit 64, a calculation unit 65, a screenrearrangement buffer 67, and a D/A conversion unit 68. Moreover, thedecoder 51 includes a frame memory 69, a selection unit 70, an intraprediction unit 71, a motion prediction/compensation unit 72, and aselection unit 73. Furthermore, the decoder 51 includes a deblockingfilter 31 b, an adaptive offset filter 81, and an ALF 82.

The accumulation buffer 61 temporarily accumulates an encoded bitstreamtransmitted from the encoder 11, and supplies the encoded bitstream tothe lossless decoding unit 62 at a predetermined timing.

The lossless decoding unit 62 receives the encoded bitstream from theaccumulation buffer 61, and decodes the encoded bitstream with a methodcorresponding to the encoding method of the lossless encoding unit 26 inFIG. 8.

Then, the lossless decoding unit 62 supplies quantization coefficientsas coded data included in a decoding result of the encoded bitstream tothe inverse quantization unit 63.

Furthermore, the lossless decoding unit 62 has a function of performingparsing. The lossless decoding unit 62 parses the necessary encodinginformation included in the decoding result of the encoded bitstream,and supplies the encoding information to the intra prediction unit 71,the motion prediction/compensation unit 72, the deblocking filter 31 b,the adaptive offset filter 81, and other necessary blocks.

The inverse quantization unit 63 performs inverse quantization on thequantization coefficients as the coded data from the lossless decodingunit 62 with a method corresponding to the quantization method of thequantization unit 25 in FIG. 8, and supplies orthogonal transformcoefficients obtained by the inverse quantization to the inverseorthogonal transform unit 64.

The inverse orthogonal transform unit 64 performs inverse orthogonaltransform on the orthogonal transform coefficients supplied from theinverse quantization unit 63 with a method corresponding to theorthogonal transform method of the orthogonal transform unit 24 in FIG.8, and supplies a residual obtained as a result of the inverseorthogonal transform, to the calculation unit 65.

To the calculation unit 65, the residual is supplied from the inverseorthogonal transform unit 64, and also a predicted image is suppliedfrom the intra prediction unit 71 or the motion prediction/compensationunit 72 via the selection unit 73.

The calculation unit 65 adds the residual from the inverse orthogonaltransform unit 64 and the predicted image from the selection unit 73together, to generate a decoded image, and supplies the decoded image tothe deblocking filter 31 b.

The screen rearrangement buffer 67 temporarily stores the decoded imagesupplied from the ALF 82, rearranges frames (pictures) of the decodedimage into the display order from the encoding (decoding) order, andsupplies the frames to the D/A conversion unit 68.

The D/A conversion unit 68 performs D/A conversion on the decoded imagesupplied from the screen rearrangement buffer 67, and outputs theconverted decoded image to a display (not illustrated) for display. Notethat, in a case where a device connected to the decoder 51 accepts animage of a digital signal, the decoder 51 can be configured withoutbeing provided with the D/A conversion unit 68.

The frame memory 69 temporarily stores the decoded image supplied fromthe ALF 82. Moreover, the frame memory 69 supplies, to the selectionunit 70, the decoded image as a reference image to be used forgenerating the predicted image, at a predetermined timing or on thebasis of an external request from the intra prediction unit 71, themotion prediction/compensation unit 72, or the like.

The selection unit 70 selects a supply destination of the referenceimage supplied from the frame memory 69. In a case where an imageencoded in the intra prediction is decoded, the selection unit 70supplies the reference image supplied from the frame memory 69 to theintra prediction unit 71. Furthermore, in a case where an image encodedin the inter-prediction is decoded, the selection unit 70 supplies thereference image supplied from the frame memory 69 to the motionprediction/compensation unit 72.

In accordance with the prediction mode included in the encodinginformation supplied from the lossless decoding unit 62, in the intraprediction mode used in the intra prediction unit 34 in FIG. 8, theintra prediction unit 71 performs intra prediction by using thereference image supplied via the selection unit 70 from the frame memory69. Then, the intra prediction unit 71 supplies the predicted imageobtained by the intra prediction to the selection unit 73.

In accordance with the prediction mode included in the encodinginformation supplied from the lossless decoding unit 62, in theinter-prediction mode used in the motion prediction/compensation unit 35in FIG. 8, the motion prediction/compensation unit 72 performsinter-prediction by using the reference image supplied via the selectionunit 70 from the frame memory 69. The inter-prediction is performedusing the motion information and the like included in the encodinginformation supplied from the lossless decoding unit 62, as necessary.

The motion prediction/compensation unit 72 supplies the predicted imageobtained by the inter-prediction to the selection unit 73.

The selection unit 73 selects the predicted image supplied from theintra prediction unit 71 or the predicted image supplied from the motionprediction/compensation unit 72, and supplies the selected predictedimage to the calculation unit 65.

The deblocking filter 31 b applies the deblocking filter to the decodedimage from the calculation unit 65 in accordance with the filterparameters included in the encoding information supplied from thelossless decoding unit 62, and supplies, to the adaptive offset filter81, the decoded image (filter image) to which the deblocking filter isapplied, or the decoded image to which the deblocking filter is notapplied.

The adaptive offset filter 81 applies the adaptive offset filter to thedecoded image from the deblocking filter 31 b as necessary in accordancewith the filter parameters included in the encoding information suppliedfrom the lossless decoding unit 62, and supplies, to the ALF 82, thedecoded image (filter image) to which the adaptive offset filter isapplied, or the decoded image to which the adaptive offset filter is notapplied.

The ALF 82 applies the ALF to the decoded image from the adaptive offsetfilter 81 as necessary, and supplies the decoded image to which the ALFis applied or the decoded image to which the ALF is not applied, to thescreen rearrangement buffer 67 and the frame memory 69.

<Decoding process>

FIG. 11 is a flowchart explaining an example of a decoding process bythe decoder 51 of FIG. 10.

In the decoding process, in step S51, the accumulation buffer 61temporarily accumulates an encoded bitstream transmitted from theencoder 11, and appropriately supplies the encoded bitstream to thelossless decoding unit 62, and the process proceeds to step S52.

In step S52, the lossless decoding unit 62 receives and decodes theencoded bitstream supplied from the accumulation buffer 61, and suppliesthe quantization coefficients as the coded data included in the decodingresult of the encoded bitstream to the inverse quantization unit 63.

Furthermore, the lossless decoding unit 62 parses the encodinginformation included in the decoding result of the encoded bitstream.Then, the lossless decoding unit 62 supplies the necessary encodinginformation to the intra prediction unit 71, the motionprediction/compensation unit 72, the deblocking filter 31 b, theadaptive offset filter 81, and other necessary blocks.

Then, the process proceeds from step S52 to step S53, and the intraprediction unit 71 or the motion prediction/compensation unit 72performs intra prediction process or inter-motion prediction process ofgenerating a predicted image, in accordance with the reference imagesupplied via the selection unit 70 from the frame memory 69, and theencoding information supplied from the lossless decoding unit 62. Then,the intra prediction unit 71 or the motion prediction/compensation unit72 supplies the predicted image obtained by the intra prediction processor the inter-motion prediction process to the selection unit 73, and theprocess proceeds from step S53 to step S54.

In step S54, the selection unit 73 selects the predicted image suppliedfrom the intra prediction unit 71 or the motion prediction/compensationunit 72, and supplies the predicted image to the calculation unit 65,and the process proceeds to step S55.

In step S55, the inverse quantization unit 63 performs inversequantization on the quantization coefficients from the lossless decodingunit 62, and supplies orthogonal transform coefficients obtained as aresult of the inverse quantization, to the inverse orthogonal transformunit 64, and the process proceeds to step S56.

In step S56, the inverse orthogonal transform unit 64 performs inverseorthogonal transform on the orthogonal transform coefficients from theinverse quantization unit 63, and supplies a residual obtained as aresult of the inverse orthogonal transform, to the calculation unit 65,and the process proceeds to step S57.

In step S57, the calculation unit 65 generates a decoded image by addingthe residual from the inverse orthogonal transform unit 64 and thepredicted image from the selection unit 73 together. Then, thecalculation unit 65 supplies the decoded image to the deblocking filter31 b, and the process proceeds from step S57 to step S58.

In step S58, the deblocking filter 31 b applies the deblocking filter tothe decoded image from the calculation unit 65 in accordance with thefilter parameters included in the encoding information supplied from thelossless decoding unit 62, and supplies a filter image obtained as aresult of the application, to the adaptive offset filter 81, and theprocess proceeds to step S59.

In step S59, the adaptive offset filter 81 applies the adaptive offsetfilter to the filter image from the deblocking filter 31 b in accordancewith the filter parameters included in the encoding information suppliedfrom the lossless decoding unit 62, and supplies a filter image obtainedas a result of the application, to the ALF 82, and the process proceedsto step S60.

The ALF 82 applies the ALF to the filter image from the adaptive offsetfilter 81, and supplies the filter image obtained as a result of theapplication, to the screen rearrangement buffer 67 and the frame memory69, and the process proceeds to step S61.

In step S61, the frame memory 69 temporarily stores the filter imagesupplied from the ALF 82, and the process proceeds to step S62. Thefilter image (decoded image) stored in the frame memory 69 is used as areference image that is a source for generating the predicted image, inthe intra prediction process or the inter-motion prediction process instep S53.

In step S62, the screen rearrangement buffer 67 performs rearrangementof the filter image supplied from the ALF 82 in the display order, andsupplies the filter image to the D/A conversion unit 68, and the processproceeds to step S63.

In step S63, the D/A conversion unit 68 performs D/A conversion on thefilter image from the screen rearrangement buffer 67, and the process,the decoding process, ends. The filter image (decoded image) after theD/A conversion is output and displayed on a display (not illustrated).

<Configuration Example of Deblocking Filter 31 a>

FIG. 12 is a block diagram illustrating a configuration example of thedeblocking filter 31 a.

Note that, the deblocking filter 31 b is configured similarly to thedeblocking filter 31 a.

In FIG. 12, the deblocking filter 31 a includes a boundary strengthcalculation unit 261, a decision unit 310, a filtering unit 320, a linebuffer 330, and a controller 340.

The boundary strength calculation unit 261 calculates bS (boundarystrength) using the color difference-related parameters related to thecolor difference, targeting the block boundary of the decoded image. Ina case where a signal in the YUV420 format is a calculation target ofthe bS, the boundary strength calculation unit 261 calculates the bS inunits of four lines in the luminance component of the decoded image,that is, in units of two lines in the color difference component of thedecoded image.

The color difference-related parameters used by the boundary strengthcalculation unit 261 to calculate the bS include a flag indicating thepresence or absence of the significant coefficient of the U component inthe block whose block boundary is located on a grid, and a flagindicating the presence or absence of the significant coefficient of theV component in the block. The color difference-related parametersincluding a flag indicating the presence or absence of the significantcoefficient of each component (Y component, U component, V component) ineach block is supplied to the boundary strength calculation unit 261from the controller 340.

The boundary strength calculation unit 261 calculates the bS using thecolor difference-related parameters and the like from the controller340. The boundary strength calculation unit 261 calculates the bS on thebasis of whether or not the significant coefficient of the colordifference component exists in two adjacent blocks sandwiching the blockboundary that is a calculation target of the bS, and the like. Theboundary strength calculation unit 261 supplies the bS to the decisionunit 310.

Note that, as the method of calculating the bS, for example, the methoddescribed in the reference document REF4 or any other method can beadopted. Furthermore, as the bS, any value representing the boundarystrength can be adopted. Here, as the bS, values 0, 1, and 2 that dividethe boundary strength into three stages are adopted, and the strongerthe boundary strength, the larger the value of the bS.

The decision unit 310 includes a filtering decision unit 311 and afiltering strength decision unit 312, and performs filtering decision.

The bS is supplied from the boundary strength calculation unit 261 tothe filtering decision unit 311. Furthermore, the decoded image issupplied to the filtering decision unit 311 from the outside of thedeblocking filter 31 a (the calculation unit 30 in FIG. 8 or thecalculation unit 65 in FIG. 10) or the line buffer 330.

The filtering decision unit 311 performs the application necessitydecision using the bS from the boundary strength calculation unit 261and, further, using the decoded image from the outside of the deblockingfilter 31 a or the line buffer 330, and the like.

The filtering decision unit 311 supplies the decision result of theapplication necessity decision to the filtering strength decision unit312.

To the filtering strength decision unit 312, the decision result of theapplication necessity decision is supplied from the filtering decisionunit 311, and also the decoded image is supplied from the outside of thedeblocking filter 31 a or the line buffer 330.

In a case where the decision result of the application necessitydecision from the filtering decision unit 311 indicates that thedeblocking filter is applied, the filtering strength decision unit 312performs the filtering strength decision for deciding the filteringstrength of the deblocking filter applied to the color differencecomponent of the decoded image by using the decoded image from theoutside of the deblocking filter 31 a or the line buffer 330. Then, thefiltering strength decision unit 312 supplies the decision result of thefiltering strength decision to the filtering unit 320 as the decisionresult of the filtering decision.

In the deblocking filter 31 a, as the filter types of the deblockingfilter applied to the color difference component of the decoded image,there are two filter types, for example, a weak filter and a chroma longfilter having a larger number of taps than the weak filter, that is,having a stronger filtering strength. The decision result of thefiltering strength indicates the weak filter or the chroma long filter.

Furthermore, in a case where the decision result of the applicationnecessity decision from the filtering decision unit 311 indicates thatthe deblocking filter is not applied, the filtering strength decisionunit 312 supplies the decision result of the application necessitydecision to the filtering unit 320 as the decision result of thefiltering decision.

To the filtering unit 320, the decision result of the filtering decisionis supplied from the filtering strength decision unit 312, and also thedecoded image is supplied from the outside of the deblocking filter 31 aor the line buffer 330.

In a case where the decision result of the filtering decision from (thefiltering strength decision unit 312 of) the decision unit 310 indicatesthat the deblocking filter is not applied, the filtering unit 320outputs the decoded image as it is without applying the deblockingfilter to the decoded image.

Furthermore, in a case where the decision result of the filteringdecision from the filtering strength decision unit 312 indicates thechroma long filter or the weak filter, the filtering unit 320 performs afiltering process of applying the chroma long filter or the weak filterindicated by the decision result of the filtering decision to thedecoded image.

That is, the filtering unit 320 performs calculation as a filteringprocess of the target pixels that are color difference pixels to besubjected to the filtering process, in the decoded image from theoutside of the deblocking filter 31 a or the line buffer 330, by usingcolor difference pixels in the vicinity of the target pixels.

Here, a pixel used for the filtering decision of the decision unit 310(a pixel referred to for the filtering decision) is also referred to asa filter reference pixel. Furthermore, a pixel used for the calculationas the filtering process of the filtering unit 320 is also referred toas a filter constituent pixel.

The filtering unit 320 outputs the color difference components obtainedby the filtering process of the target pixels as the color differencecomponents of the filter pixels (the pixels constituting the filterimage after the filtering process).

A decoded image is supplied to the line buffer 330 from the outside ofthe deblocking filter 31 a. The line buffer 330 appropriately stores thecolor difference components of the decoded image from the outside of thedeblocking filter 31 a. Note that, the line buffer 330 has a storagecapacity for storing the color difference components for a predeterminednumber of lines (number of rows), and when the color differencecomponents for the storage capacity are stored, a new color differencecomponent is stored in the form of being overwritten on the oldest colordifference component.

Here, it is assumed that the deblocking filter 31 a processes thedecoded image in the order of raster scan.

In the deblocking filter 31 a, the process is performed in units of apredetermined block (which may be, for example, a block of a unit inwhich orthogonal transform is performed, or a block including a unit inwhich orthogonal transform is performed). In the deblocking filter 31 a,for example, a plurality of blocks such as those for one line can beprocessed in the order of raster scan, and can also be processed inparallel.

The decision unit 310 and the filtering unit 320 include a built-ininternal buffer having a capacity capable of storing color differencecomponents of a line in the horizontal direction included in a targetblock that is a block to be processed by the deblocking filter 31 a. Thedecision unit 310 and the filtering unit 320 store the color differencecomponents of the line in the horizontal direction included in thetarget block in the internal buffer, and use the color differencecomponents stored in the internal buffer as color difference componentsof the filter reference pixel and the filter constituent pixel, toprocess the target block.

In a case where the deblocking filter 31 a is applied to the horizontalblock boundary on the upper side of the target block, color differencecomponents of pixels in the target block and color difference componentsof pixels in a block adjacent to the upper side of the target block arerequired.

The color difference components of the pixels in the target block arestored in the internal buffer when the target block is processed. On theother hand, since the color difference components of the pixels in theblock adjacent to the upper side of the target block are not the colordifference components of the pixels in the target block, the values arenot stored in the internal buffer when the target block is processed.

Thus, the line buffer 330 stores color difference components of pixelsof a line (pixels belonging a line) necessary for applying thedeblocking filter 31 a to the horizontal block boundary on the upperside of the target block among lines in the horizontal directionincluded in the block adjacent to the upper side of the target block.The pixels of the line necessary for applying the deblocking filter 31 aare pixels that are used for the filter reference pixel and the filterconstituent pixel.

The controller 340 controls each block constituting the deblockingfilter 31 a. Furthermore, the controller 340 acquires colordifference-related parameters and the like necessary for calculating thebS by performing generation or the like, and supplies the parameters tothe boundary strength calculation unit 261.

Note that, in the present embodiment, it is assumed that the deblockingfilter 31 a processes, for example, the decoded images in the order ofraster scan. However, the deblocking filter 31 a can perform the decodedimage in an order other than the order of raster scan. For example, thedeblocking filter 31 a can repeat processing the decoded image from topto bottom, from left to right. In this case, the horizontal (lateral)(left and right) and vertical (longitudinal) (up and down) describedbelow are reversed (swapped).

FIG. 13 is a flowchart explaining the process of the deblocking filter31 a of FIG. 12.

In the deblocking filter 31 a, the line buffer 330 appropriately storesthe color difference components of the decoded image supplied from theoutside of the deblocking filter 31 a.

Then, in step S101, the boundary strength calculation unit 261calculates the bS for the block boundary located on the grid andsupplies the bS to the decision unit 310, and the process proceeds tostep S102.

In steps S102 to S104, the decision unit 310 performs the filteringdecision for each partial block boundary (partial vertical blockboundary and partial horizontal block boundary).

That is, in step S102, the decision unit 310 decides whether or not acondition 1 described later is satisfied.

In a case where it is decided in step S102 that the condition 1 is notsatisfied, the decision unit 310 decides that the deblocking filter 31 ais not applied, and the filtering unit 320 does not perform thefiltering process to (pixels of a line orthogonal to) the partial blockboundary for which it is decided that the condition 1 is not satisfied,and the process ends.

Furthermore, in a case where it is decided in step S102 that thecondition 1 is satisfied, the process proceeds to step S103, and thedecision unit 310 decides whether or not a condition 2 described lateris satisfied.

In a case where it is decided in step S103 that the condition 2 is notsatisfied, the decision unit 310 decides that the deblocking filter 31 ais not applied, and the filtering unit 320 does not perform thefiltering process to the partial block boundary for which it is decidedthat the condition 2 is not satisfied, and the process ends.

Furthermore, in a case where it is decided in step S103 that thecondition 2 is satisfied, the process proceeds to step S104, and thedecision unit 310 decides whether or not a condition 3 described lateris satisfied.

In a case where it is decided in step S104 that the condition 3 is notsatisfied, the process proceeds to step S105, and the decision unit 310decides that the weak filter is applied. Then, the filtering unit 320performs a filtering process of the weak filter to the partial blockboundary for which it is decided that the condition 3 is not satisfied,and the process ends.

Furthermore, in a case where it is decided in step S104 that thecondition 3 is satisfied, the process proceeds to step S106, and thedecision unit 310 decides that the chroma long filter is applied. Then,the filtering unit 320 performs a filtering process of the chroma longfilter to the partial block boundary for which it is decided that thecondition 3 is satisfied, and the process ends.

In FIG. 13, for example, the decision in steps S102 and S103 correspondsto the adaptation necessity decision, and the decision in step S104corresponds to the filtering strength decision.

Note that, even in a case where it is decided in step S103 that thecondition 2 is not satisfied, when the bS is 2, which indicates that theboundary strength is the strongest, it can be decided in the decisionunit 310 that the weak filter is applied. Then, in the filtering unit320, the filtering process of the weak filter can be performed on thepartial block boundary whose bS is 2 although it is decided that thecondition 2 is not satisfied.

<Filtering Decision of YUV420 Format>

FIG. 14 is a diagram explaining filtering decision in a case where thecolor format is the YUV420 format.

That is, FIG. 14 is a diagram explaining the filtering decision(vertical block boundary filtering decision) for the partial verticalblock boundary of the decoded image in the YUV420 format.

In the YUV420 format, if the partial vertical block boundary of theluminance component is the vertical block boundary for four lines in thehorizontal direction, the partial vertical block boundary of the colordifference component is ½ of the partial vertical block boundary of theluminance component, that is, a vertical block boundary for two lines inthe horizontal direction.

For example, if the partial vertical block boundary of the luminancecomponent is a partial vertical block boundary b as a combined portionof the partial vertical block boundaries b1 and b2 illustrated in FIG.3, the vertical block boundary of the color difference component is thepartial vertical block boundary b1 and the partial vertical blockboundary b2.

In this case, the length of the partial vertical block boundary of theluminance component is four pixels (for four lines), and the length ofthe partial vertical block boundary of the color difference component istwo pixels (for two lines).

For the luminance component in the YUV420 format, the deblocking filter31 a performs the vertical block boundary filtering decision for thepartial vertical block boundary of the luminance component by using theluminance components of pixels of two lines, the first line and thefourth line, which are (horizontal) lines located at both ends of thepartial vertical block boundary of the luminance component, out of fourlines in the horizontal direction of the luminance component orthogonalto the partial vertical block boundary of the luminance component.

The vertical block boundary filtering decision for the partial verticalblock boundary of the luminance component here is filtering decision fordeciding whether or not a deblocking filter is applied to the luminancecomponents of pixels of four (horizontal) lines, the first line to thefourth line, orthogonal to the partial vertical block boundary of theluminance component.

Furthermore, the deblocking filter 31 a performs the vertical blockboundary filtering decision for the partial vertical block boundary ofthe color difference component by using the color difference componentof the color difference pixel of the first line of two lines in thehorizontal direction of the color difference component orthogonal to thepartial vertical block boundary of the color difference component.

The vertical block boundary filtering decision for the partial verticalblock boundary of the color difference component here is filteringdecision for deciding whether or not a deblocking filter is applied tothe color difference components of pixels of two (horizontal) lines, thefirst line to the second line, orthogonal to the partial vertical blockboundary of the color difference component.

For the YUV420 format, the truth or falsehood (1 or 0) of the equation(14) is decided, as the condition 1, in the vertical block boundaryfiltering decision of the color difference component.

(bS==2∥(bS==1 && Large block decision))

Large block decision: (EDGE_VER && block_width>8)∥(EDGE_HOR &&block_height>8)   (14)

Here, the bS in the equation (14) is the bS calculated from two adjacentblocks sandwiching the partial vertical block boundary. EDGE VER is true(1) in a case where the partial block boundary that is a target of thefiltering decision is a partial vertical block boundary, and false (0)otherwise (in a case where it is a partial horizontal block boundary).EDGE HOR is true in a case where the partial block boundary that is atarget of the filtering decision is a partial horizontal block boundary,and false otherwise (in a case where it is a partial vertical blockboundary).

For the YUV420 format, as the condition 2, the truth or falsehood of theequation (15) is decided.

d<(beta>>1)   (15)

A>>B represents that A is shifted to the right by B bits.

The d in the equation (15) is calculated in accordance with theequations (16) to (19).

dp0=Abs(p _(2, 0)−2*p _(1, 0) +p _(0, 0))   (16)

dq0=Abs(q _(2, 0)−2*q _(1, 0) +q _(0, 0))   (17)

dpq0=dp0+dq0   (18)

d=dpq0   (19)

Here, in the filtering decision of the reference document REF4, thedecision of the condition C92 similar to that of the Non-Patent Document1 is performed. In calculation of the d of the condition C92, asindicated in the equations (1) to (7), the color difference componentsp_(2, 0), p_(1, 0), p_(0, 0), q_(2, 0), q_(1, 0), and q_(0, 0), andp_(2, 1), p_(1, 1), p_(0, 1), q_(2, 1), q_(1, 1), and q_(0, 1) are usedof the color difference pixels of the two lines L11 and L12 of the colordifference component orthogonal to the partial vertical block boundaryb1 (FIG. 3).

On the other hand, in the vertical block boundary filtering decision ofthe deblocking filter 31 a, for the color difference component, incalculation of the d of the condition 2, as indicated in the equations(16) to (19) , the color difference components p_(2, 0), p_(1, 0),p_(0, 0), q_(2, 0), q_(1, 0), and q_(0, 0) are used of only the colordifference pixels of the first line L11 of the two lines L11 and L12 ofthe color difference component orthogonal to the partial vertical blockboundary b1.

For that reason, the equations (16) to (19) are equations obtained bydeleting, from the equations (1) to (7), portions related to the colordifference components p_(2, 1), p_(1, 1), p_(0, 1), q_(2, 1), q_(1, 1),and q_(0, 1) of the color difference pixels of the second line L12.

For the YUV420 format, as the condition 3, the truth or falsehood of theequation (20) is decided.

xUseStrongFiltering(LinePos-#0)   (20)

LinePos-#j−1 represents the j-th line of the two lines L11 and L12 ofthe color difference component orthogonal to the partial vertical blockboundary b1.

The function xUseStrongFiltering(LinePos-#j−1) in the equation (20) is afunction similar to that used for the filtering decision of theluminance component, and returns a value of the truth or falsehood (1 or0) depending on whether or not the equations (21) to (23) are satisfied.

|p3−p0|+|q3−q0|<(beta>>3)   (21)

|p2−2*p1+p0|+|q2−2*q1+q0|<(beta >>2)   (22)

|p0−q0|<((tc*5+1)>>1)   (23)

In the equations (21) to (23), p_(i) and q_(k) represent the colordifference components of the color difference pixels p_(i, j) andq_(k, j) of the i-th and k-th columns from the partial vertical blockboundary b1, in the j-th row of the two adjacent blocks Bp and Bqsandwiching the partial vertical block boundary b1, and the index j ofp_(i, j) and q_(k, j) is omitted.

Furthermore, tc is a parameter given depending on the quantizationparameter.

|1p3 −p0|+|q3−q0| in the equation (21) represents flatness of thepartial vertical block boundary b1. |p2−2*p1+p0|+|q2−2*q1+q0| in theequation (22) represents continuity of the partial vertical blockboundary b1. |p0−q0| in the equation (23) represents a gap at thepartial vertical block boundary bl.

Here, in the filtering decision of the color difference component of thereference document REF4, the truth or falsehood of the equation (24) isdecided.

xUseStrongFiltering(LinePos-#0) && xUseStrongFiltering(LinePos-#1)  (24)

In decision of the truth or falsehood of the equation (24), the colordifference components p_(3, 0), p_(2, 0), p_(1, 0), p_(0, 0), q_(3, 0),q_(2, 0), q_(1, 0), and q_(0, 0), and p_(3, 1), p_(2, 1), p_(1, 1),p_(0, 1), q_(3, 1), q_(2, 1), q_(1, 1), and q_(0, 1) are used of thecolor difference pixels of the two lines L11 and L12 of the colordifference component orthogonal to the partial vertical block boundaryb1.

On the other hand, in the filtering decision of the color differencecomponent of the deblocking filter 31 a, in decision of the truth orfalsehood of the condition 3 of the equation (20), the color differencecomponents p_(3, 0), p_(2, 0), p_(1, 0), p_(0, 0), q_(3, 0), q_(2, 0),q_(1, 0), and q_(0, 0) are used of only the color difference pixels ofthe first line L11 of the two lines L11 and L12 of the color differencecomponent orthogonal to the partial vertical block boundary b1.

For that reason, the equation (20) is an equation obtained by deleting,from the equation (24), a portion xUseStrongFiltering(LinePos-#1)related to the color difference components p_(3, 1), p_(2, 1), p_(1, 1),p_(0, 1), q_(2, 1), q_(1, 1), and q_(0, 1) of the color differencepixels of the second line L12.

For the YUV420 format, the filtering decision (horizontal block boundaryfiltering decision) for the partial horizontal block boundary of thedecoded image is performed similarly to the vertical block boundaryfiltering decision, and thus the description thereof will be omitted.

Note that, here, for the color difference component of the YUV420format, the vertical block boundary filtering decision is performed byusing the color difference component of the color difference pixels ofthe first line of the two lines in the horizontal direction of the colordifference component orthogonal to the partial vertical block boundaryof the color difference component.

For the color difference component of the YUV420 format, the verticalblock boundary filtering decision can be performed by using the colordifference components of the color difference pixels of the second line,not the first line of the two lines in the horizontal direction of thecolor difference component orthogonal to the partial vertical blockboundary of the color difference component. The same applies to thehorizontal block boundary filtering decision. <Filtering Decision ofYUV444 Format>

FIG. 15 is a diagram explaining filtering decision in a case where thecolor format is the YUV444 format.

That is, FIG. 15 is a diagram explaining the vertical block boundaryfiltering decision for the partial vertical block boundary of thedecoded image in the YUV444 format.

In the YUV444 format, if the partial vertical block boundary of theluminance component is the vertical block boundary for four lines in thehorizontal direction, the partial vertical block boundary of the colordifference component is a vertical block boundary for four lines in thehorizontal direction, similarly to the partial vertical block boundaryof the luminance component.

For example, the partial vertical block boundaries of the luminancecomponent and the color difference component both are the partialvertical block boundary b as the combined portion of the partialvertical block boundaries b1 and b2 illustrated in FIG. 3.

In this case, the lengths of the partial vertical block boundaries ofthe luminance component and the color difference component both are fourpixels (for four lines).

For the luminance component in the YUV444 format, the deblocking filter31 a performs the vertical block boundary filtering decision for thepartial vertical block boundary of the luminance component by using theluminance components of pixels of two lines, the first line and thefourth line, which are (horizontal) lines located at both ends of thepartial vertical block boundary of the luminance component, out of fourlines in the horizontal direction of the luminance component orthogonalto the partial vertical block boundary of the luminance component.

The vertical block boundary filtering decision for the partial verticalblock boundary of the luminance component here is filtering decision fordeciding whether or not a deblocking filter is applied to the luminancecomponents of pixels of four lines, the first line to the fourth line,orthogonal to the partial vertical block boundary of the luminancecomponent.

Furthermore, the deblocking filter 31 a performs the vertical blockboundary filtering decision for the partial vertical block boundary ofthe color difference component by using the color difference componentsof the color difference pixels of a line identical to a line used whenperforming the vertical block boundary filtering decision that decideswhether or not a deblocking filter is applied to the luminancecomponent.

That is, the deblocking filter 31 a performs the vertical block boundaryfiltering decision for the partial vertical block boundary of the colordifference component by using the color difference components of thecolor difference pixels of two lines, the first line and the fourthline, which are (horizontal) lines located at both ends of the partialvertical block boundary of the color difference component, out of fourlines in the horizontal direction of the color difference componentorthogonal to the partial vertical block boundary of the colordifference component.

The vertical block boundary filtering decision for the partial verticalblock boundary of the color difference component here is filteringdecision for deciding whether or not a deblocking filter is applied tothe color difference components of pixels of four lines, the first lineto the fourth line, orthogonal to the partial vertical block boundary ofthe color difference component.

For the YUV444 format, in the vertical block boundary filtering decisionof the color difference component, as the condition 1, the truth orfalsehood of the equation (14) is decided, similarly to the YUV420format.

For the YUV444 format, as the condition 2, the truth or falsehood of theequation (25) is decided.

d<beta   (25)

The d in the equation (25) is calculated in accordance with theequations (26) to (32).

dp0=Abs(p2, 0−2*p1, 0+p0, 0)   (26)

dp3=Abs(p2, 3−2*p1, 3+p0, 3)   (27)

qp0=Abs(q2, 0−2*q1, 0+q0, 0)   (28 )

qp3=Abs(q2, 3−2*q1, 3+q0, 3)   (29)

dpq0=dp0+dq0   (30)

dpq3=dp3+dq3   (31)

d=dpq0+dpq3   (32)

In the vertical block boundary filtering decision of the deblockingfilter 31 a, for the color difference component, in calculation of the din the condition 2, as indicated in the equations (26) to (32), out ofthe four lines L11, L12, L21, and L22 of the color difference componentorthogonal to the partial vertical block boundary b, the colordifference components p_(2, 0), p_(1, 0), p_(0, 0), q_(2, 0), q_(1, 0),and q_(0, 0) of the color difference pixels of the first line L11, andthe color difference components p_(2, 3), p_(1, 3), p_(0, 3), q_(2, 3),q_(1, 3), and q_(0, 3) of the color difference pixels of the fourth lineL22 are used.

For the YUV444 format, as the condition 3, the truth or falsehood of theequation (33) is decided.

xUseStrongFiltering(LinePos-#0) && xUseStrongFiltering(LinePos-#3)  (33)

The function xUseStrongFiltering(LinePos-#j−1) returns a value of thetruth or falsehood depending on whether or not the equations (21) to(23) are satisfied, as described in FIG. 14.

Thus, in the vertical block boundary filtering decision of thedeblocking filter 31 a, in decision of the truth or falsehood of thecondition 3 of the equation (33), out of the four lines L11, L12, L21,and L22 of the color difference component orthogonal to the partialvertical block boundary b, the color difference components p_(3, 0),p_(2, 0), p_(1, 0), p_(0, 0), q_(3, 0), q_(2, 0), q_(1, 0), and q_(0, 0)of the color difference pixels of the first line L11, and the colordifference components p_(3, 3), p_(2, 3), p_(1, 3), p_(0, 3), q_(3, 3),q_(2, 3), q_(1, 3), and q_(0, 3) of the color difference pixels of thefourth line L22 are used.

For the YUV444 format, the filtering decision (horizontal block boundaryfiltering decision) for the partial horizontal block boundary of thedecoded image is performed similarly to the vertical block boundaryfiltering decision, and thus the description thereof will be omitted.

Note that, here, for the luminance component and color differencecomponent of the YUV444 format, the vertical block boundary filteringdecision is performed by using the pixels of the first line and thefourth line of the four lines in the horizontal direction orthogonal tothe partial vertical block boundary.

For the YUV444 format, the vertical block boundary filtering decisioncan be performed by using pixels of any one or more lines other than thefirst line and fourth line of the four lines in the horizontal directionorthogonal to the partial vertical block boundary. However, in thevertical block boundary filtering decision of the color differencecomponent, pixels are used of the same line used in the vertical blockboundary filtering decision of the luminance component. The same appliesto the horizontal block boundary filtering decision.

<Filtering Decision of YUV422 Format>

FIG. 16 is a diagram explaining filtering decision in a case where thecolor format is the YUV422 format.

That is, FIG. 16 is a diagram explaining the vertical block boundaryfiltering decision for the partial vertical block boundary of thedecoded image in the YUV422 format.

In the YUV422 format, if the partial vertical block boundary of theluminance component is the vertical block boundary for four lines in thehorizontal direction, the partial vertical block boundary of the colordifference component is a vertical block boundary for four lines in thehorizontal direction, similarly to the partial vertical block boundaryof the luminance component.

For example, the partial vertical block boundaries of the luminancecomponent and the color difference component both are the partialvertical block boundary b as the combined portion of the partialvertical block boundaries b1 and b2 illustrated in FIG. 3.

In this case, the lengths of the partial vertical block boundaries ofthe luminance component and the color difference component both are fourpixels (for four lines).

For the luminance component in the YUV422 format, the deblocking filter31 a performs the vertical block boundary filtering decision for thepartial vertical block boundary of the luminance component by using theluminance components of pixels of two lines, the first line and thefourth line, which are (horizontal) lines located at both ends of thepartial vertical block boundary of the luminance component, out of fourlines in the horizontal direction of the luminance component orthogonalto the partial vertical block boundary of the luminance component.

The vertical block boundary filtering decision for the partial verticalblock boundary of the luminance component here is filtering decision fordeciding whether or not a deblocking filter is applied to the luminancecomponents of pixels of four lines, the first line to the fourth line,orthogonal to the partial vertical block boundary of the luminancecomponent.

Furthermore, the deblocking filter 31 a performs the vertical blockboundary filtering decision for the partial vertical block boundary ofthe color difference component by using the color difference componentsof the color difference pixels of a line identical to a line used whenperforming the vertical block boundary filtering decision that decideswhether or not a deblocking filter is applied to the luminancecomponent.

That is, the deblocking filter 31 a performs the vertical block boundaryfiltering decision for the partial vertical block boundary of the colordifference component by using the color difference components of thecolor difference pixels of two lines, the first line and the fourthline, which are (horizontal) lines located at both ends of the partialvertical block boundary of the color difference component, out of fourlines in the horizontal direction of the color difference componentorthogonal to the partial vertical block boundary of the colordifference component.

The vertical block boundary filtering decision for the partial verticalblock boundary of the color difference component here is filteringdecision for deciding whether or not a deblocking filter is applied tothe color difference components of pixels of four lines, the first lineto the fourth line, orthogonal to the partial vertical block boundary ofthe color difference component.

For the YUV422 format, in the vertical block boundary filtering decisionof the color difference component, as the condition 1, the truth orfalsehood of the equation (14) is decided, similarly to the YUV420format.

For the YUV422 format, as the condition 2, the truth or falsehood of theequations (34) and (35) is decided.

edgeDir==EDGE_VER   (34)

d<beta   (35)

In edgeDir, EDGE_VER is set in a case where the partial block boundarythat is a target of the filtering decision is the partial vertical blockboundary, and EDGE_HOR is set in a case where the partial block boundarythat is the target of the filtering decision is the partial horizontalblock boundary.

The d in the equation (35) is calculated in accordance with theequations (26) to (32) described in FIG. 15.

In the vertical block boundary filtering decision of the deblockingfilter 31 a, for the color difference component, in calculation of the din the condition 2, as indicated in the equations (26) to (32), out ofthe four lines L11, L12, L21, and L22 of the color difference componentorthogonal to the partial vertical block boundary b, the colordifference components p_(2, 0), p_(1, 0), p_(0, 0), q_(2, 0), q_(1, 0),and q_(0, 0) of the color difference pixels of the first line L11, andthe color difference components p_(2, 3), p_(1, 3), p_(0, 3), p_(2, 3),q_(1, 3), and q_(0, 3) of the color difference pixels of the fourth lineL22 are used.

For the YUV422 format, as the condition 3, the truth or falsehood of theequation (33) described in FIG. 15 is decided.

Thus, for the YUV422 format, in the vertical block boundary filteringdecision of the deblocking filter 31 a, similarly to the case of theYUV444 described in FIG. 15, in decision of the truth or falsehood ofthe condition 3 of the equation (33), out of the four lines L11, L12,L21, and L22 of the color difference component orthogonal to the partialvertical block boundary b, the color difference components p_(3, 0),p_(2, 0), p_(1, 0), p_(0, 0), q_(3, 0), q_(2, 0), q_(1, 0), and q_(0, 0)of the color difference pixels of the first line L11, and the colordifference components p_(3, 3), p_(2, 3), p_(1, 3), p_(0, 3), q_(3, 3),q_(2, 3), q_(1, 3), and g_(0, 3) of the color difference pixels of thefourth line L22 are used.

Note that, here, for the luminance component and color differencecomponent of the YUV422 format, the vertical block boundary filteringdecision is performed by using the pixels of the first line and thefourth line of the four lines in the horizontal direction orthogonal tothe partial vertical block boundary.

For the YUV422 format, the vertical block boundary filtering decisioncan be performed by using pixels of any one or more lines other than thefirst line and fourth line of the four lines in the horizontal directionorthogonal to the partial vertical block boundary. However, in thevertical block boundary filtering decision of the color differencecomponent, pixels are used of the same line used in the vertical blockboundary filtering decision of the luminance component.

FIG. 17 is a diagram explaining filtering decision in a case where thecolor format is the YUV422 format.

That is, FIG. 17 is a diagram explaining the filtering decision(horizontal block boundary filtering decision) for the partialhorizontal block boundary of the decoded image in the YUV422 format.

Note that, in FIG. 17, a description will be given assuming that theblock boundary BB is not a vertical block boundary but a horizontalblock boundary in FIG. 3. For example, it is assumed that the block Bpand the block Bq are the blocks above and below the (horizontal) blockboundary BB, respectively. In this case, as described in FIG. 3, inp_(i, j) and q_(k, j), i and k are row indexes and j is a column index.

Furthermore, the partial block boundaries b1, b2, and b are partialhorizontal block boundaries.

In the YUV422 format, assuming that the partial horizontal blockboundary of the luminance component is the horizontal block boundary forfour lines in the vertical direction, the partial horizontal blockboundary of the color difference component is ½ of the partialhorizontal block boundary of the luminance component, that is, thehorizontal block boundary for two lines in the vertical direction.

For example, if the partial horizontal block boundary of the luminancecomponent is the partial horizontal block boundary b as the combinedportion of the partial horizontal block boundaries b1 and b2 illustratedin FIG. 3, the horizontal block boundary of the color differencecomponent is the partial horizontal block boundary b1 and the partialhorizontal block boundary b2.

In this case, the length of the partial horizontal block boundary of theluminance component is four pixels (for four lines), and the length ofthe partial horizontal block boundary of the color difference componentis two pixels (for two lines).

For the luminance component in the YUV422 format, the deblocking filter31 a performs the horizontal block boundary filtering decision for thepartial horizontal block boundary of the luminance component by usingthe luminance components of pixels of two lines, the first line and thefourth line, which are (vertical) lines located at both ends of thepartial horizontal block boundary of the luminance component, out offour lines in the vertical direction of the luminance componentorthogonal to the partial horizontal block boundary of the luminancecomponent.

The horizontal block boundary filtering decision for the partialhorizontal block boundary of the luminance component here is filteringdecision for deciding whether or not a deblocking filter is applied tothe luminance component of pixels of four (vertical) lines, the firstline to the fourth line, orthogonal to the partial horizontal blockboundary of the luminance component.

Furthermore, the deblocking filter 31 a performs the horizontal blockboundary filtering decision for the partial horizontal block boundary ofthe color difference component by using the color difference componentof the color difference pixel of the first line of two lines in thevertical direction of the color difference component orthogonal to thepartial horizontal block boundary of the color difference component.

The horizontal block boundary filtering decision for the partialhorizontal block boundary of the color difference component here isfiltering decision for deciding whether or not a deblocking filter isapplied to the color difference component of pixels of two (vertical)lines, the first line to the second line, orthogonal to the partialhorizontal block boundary of the color difference component.

For the YUV422 format, in the horizontal block boundary filteringdecision of the color difference component, as the condition 1, thetruth or falsehood of the equation (14) is decided, similarly to theYUV420 format. However, for the YUV422 format, the bS of the equation(14) calculated in the horizontal block boundary filtering decision ofthe color difference component is the bS calculated from two adjacentblocks sandwiching the partial horizontal block boundary.

For the YUV422 format, in the horizontal block boundary filteringdecision of the color difference component, as the condition 2, thetruth or falsehood of the equations (36) and (37) is decided.

edgeDir==EDGE_HOR   (36)

d<(beta>>1)   (37)

In edgeDir, EDGE_VER is set in a case where the partial block boundarythat is a target of the filtering decision is the partial vertical blockboundary, and EDGE_HOR is set in a case where the partial block boundarythat is the target of the filtering decision is the partial horizontalblock boundary.

The d in the equation (37) is calculated in accordance with theequations (16) to (19) described in FIG. 14.

In the horizontal block boundary filtering decision of the deblockingfilter 31 a, for the color difference component, in calculation of the dof the condition 2, as indicated in the equations (16) to (19), thecolor difference components p_(2, 0), p_(1, o), p_(0, 0), q_(2, 0),q_(1, 0), and q_(0, 0) are used of only the color difference pixels ofthe first line L11 of the two lines L11 and L12 of the color differencecomponent orthogonal to the partial horizontal block boundary b1.

For the YUV422 format, in the horizontal block boundary filteringdecision of the color difference component, as the condition 3, thetruth or falsehood of the equation (20) is decided.

Thus, in the filtering decision of the color difference component of thedeblocking filter 31 a, for the color difference component, in decisionof the truth or falsehood of the condition 3 of the equation (20), thecolor difference components p_(3, 0), p_(2, 0), p_(1, 0), p_(0, 0),q_(3, 0), q_(2, 0), q_(1, 0), and q_(0, 0) are used of only the colordifference pixels of the first line L11 of the two lines L11 and L12 ofthe color difference component orthogonal to the partial horizontalblock boundary b1.

Note that, here, for the color difference component of the YUV422format, the horizontal block boundary filtering decision is performed byusing the color difference component of the color difference pixels ofthe first line of the two lines in the vertical direction of the colordifference component orthogonal to the partial horizontal block boundaryof the color difference component.

For the color difference component of the YUV422 format, the horizontalblock boundary filtering decision can be performed by using the colordifference components of the color difference pixels of the second line,not the first line of the two lines in the vertical direction of thecolor difference component orthogonal to the partial horizontal blockboundary of the color difference component.

Note that, here, although the length of the partial block boundary ofthe luminance component is set to four pixels, the number of pixels (thenumber of lines) exceeding four pixels can be adopted as the length ofthe partial block boundary of the luminance component.

For example, as the length of the partial block boundary of theluminance component, 8 pixels, 16 pixels, or the like can be adopted.

In a case where a length of eight pixels is adopted as the length of thepartial block boundary of the luminance component, for example, thelengths of the partial horizontal block boundary and the partialvertical block boundary of the color difference component in the YUV444format, and the partial vertical block boundary in the YUV422 format areeight pixels similar to the case of the luminance component.Furthermore, the lengths of the partial horizontal block boundary in theYUV422 format and the partial horizontal block boundary and the partialvertical block boundary of the color difference component of the YUV420format are four pixels, which is ½ of the case of the luminancecomponent.

Moreover, the vertical block boundary filtering decision of theluminance component and the color difference component in the YUV422format can be performed by using pixels of two lines adjacent to eachother in the center of the partial vertical block boundary, in additionto the pixels of the lines located at both ends of the partial verticalblock boundary among the lines orthogonal to the partial vertical blockboundary.

For example, in a case where the length of the partial vertical blockboundary of the luminance component and color difference component inthe YUV422 format is eight pixels, it is possible to perform thevertical block boundary filtering decision that decides whether or not adeblocking filter is applied to pixels of each of the luminancecomponent and color difference component of eight lines orthogonal tothe partial vertical block boundary, by using the luminance componentand color difference component of pixels of four lines, the first,fourth, fifth, and eighth lines, out of the eight lines orthogonal tothe partial vertical block boundary.

The same applies to the horizontal block boundary filtering decision andthe vertical block boundary filtering decision of the luminancecomponent and the color difference component of the YUV444 format in theabove points.

<Others>

(Application Target of the Present Technology)

The present technology can be applied to any image coding/decodingmethod. That is, unless inconsistent with the present technologydescribed above, the specifications of various processes related toimage coding/decoding, such as transform (inverse transform),quantization (inverse quantization), coding (decoding), and prediction,are arbitrary, and are not limited to the examples described above.Furthermore, unless inconsistent with the present technology describedabove, some of these processes may be omitted.

(Block)

Furthermore, in the present specification, “block” (not a blockindicating a processing unit) used for description as a partial area ora unit of processing of an image (picture) indicates an arbitrarypartial area in a picture, unless otherwise specified, and the size,shape, characteristics, and the like are not limited. For example, the“block” includes arbitrary partial areas (units of processing) such asthe transform block (TB), transform unit (TU), prediction block (PB),prediction unit (PU), smallest coding unit (SCU), coding unit (CU),largest coding unit (LCU), coding tree block (CTB), coding tree unit(CTU), transform block, sub-block, macroblock, tile, or slice describedin the reference documents REF1 to REF3 and the like.

(Unit of Processing)

A unit of data in which the various types of information described aboveis set, and a unit of data targeted by the various processes each arearbitrary and are not limited to the examples described above. Forexample, these information and processes each may be set for eachTransform Unit (TU), Transform Block (TB), Prediction Unit (PU),Prediction Block (PB), Coding Unit (CU), Largest Coding Unit (LCU),sub-block, block, tile, slice, picture, sequence, or component, or datain units of data of those may be targeted. Of course, the unit of datacan be set for each piece of information or process, and it is notnecessary that the units of data of all the information and processesare unified. Note that, a storage location of these pieces ofinformation is arbitrary, and may be stored in the header, parameterset, or the like of the unit of data described above. Furthermore, thosemay be stored in a plurality of locations.

(Control Information)

Control information related to the present technology described abovemay be transmitted from the coding side to the decoding side. Forexample, control information (for example, enabled_flag) may betransmitted that controls whether or not the application of the presenttechnology described above is permitted (or prohibited). Furthermore,for example, control information may be transmitted indicating an objectto which the present technology is applied (or an object to which thepresent technology is not applied). For example, control information maybe transmitted that specifies the block size (upper limit, lower limit,or both), frame, component, layer, or the like to which the presenttechnology is applied (or for which application is permitted orprohibited).

(Block Size Information)

In specification of the size of the block to which the presenttechnology is applied, the block size may not only be directlyspecified, but also be specified indirectly. For example, the block sizemay be specified by using identification information for identifying thesize. Furthermore, for example, the block size may be specified by aratio to or a difference from the size of a reference block (forexample, the LCU, the SCU, and the like). For example, in a case whereinformation for specifying the block size is transmitted as a syntaxelement or the like, the information for indirectly specifying the sizeas described above may be used as the information. By doing so, theamount of information can be reduced, and the coding efficiency can beimproved in some cases. Furthermore, the specification of the block sizealso includes specification of a block size range (for example,specification of an allowable block size range, or the like).

(Others)

Note that, in the present specification, the “flag” is information foridentifying a plurality of states, and includes not only informationused for identifying two states of true (1) or false (0), but alsoinformation capable of identifying three or more states. Thus, valuesthat can be taken by the “flag” may be, for example, two values of 1/0,or three or more values. That is, the number of bits constituting the“flag” is arbitrary, and may be 1 bit or a plurality of bits.Furthermore, the identification information (including the flag) isassumed to include not only the identification information in thebitstream but also difference information of the identificationinformation with respect to a certain reference information in thebitstream, so that the “flag” and “identification information” includenot only the information but also the difference information withrespect to the reference information, in the present specification.

Furthermore, various types of information (metadata, and the like)regarding the coded data (bitstream) may be transmitted or recorded inany form as long as those are associated with the coded data. Here, aterm “associate” means that, for example, when processing one data, theother data is made to be usable (linkable). That is, the data associatedwith each other may be collected as one data, or may be individual data.For example, information associated with coded data (image) may betransmitted on a transmission line different from that for the codeddata (image). Furthermore, for example, the information associated withthe coded data (image) may be recorded in a recording medium differentfrom that for the coded data (image) (or in a different recording areaof the same recording medium). Note that, this “association” may be apart of data, not the entire data. For example, an image and informationcorresponding to the image may be associated with each other in anarbitrary unit such as a plurality of frames, one frame, or a portionwithin a frame.

Note that, in this specification, terms “combine”, “multiplex”, “add”,“integrate”, “include”, “store”, “put in”, “enclose”, “insert”, and thelike mean to combine a plurality of objects into one, for example, tocombine coded data and metadata into one, and the terms mean one methodof the “associate” described above.

The present technology can also be implemented as any configurationconstituting a device or system, for example, a processor as a systemlarge scale integration (LSI) or the like, a module using a plurality ofprocessors and the like, a unit using a plurality of modules and thelike, a set in which other functions are further added to the unit, orthe like (that is, a configuration of a part of the device).

<Description of Computer to which the Present Technology is Applied>

Next, a series of processes described above can be performed by hardwareor software. In a case where the series of processes is performed bysoftware, a program constituting the software is installed in ageneral-purpose computer or the like.

FIG. 18 is a block diagram illustrating a configuration example of anembodiment of a computer in which a program for executing the series ofprocesses described above is installed.

The program can be recorded in advance on a hard disk 905 or a ROM 903as a recording medium incorporated in the computer.

Alternatively, the program can be stored (recorded) in a removablerecording medium 911 driven by a drive 909. Such a removable recordingmedium 911 can be provided as so-called packaged software. Here,examples of the removable recording medium 911 include a flexible disk,a Compact Disc Read Only Memory (CD-ROM), a Magneto Optical (MO) disk, aDigital Versatile Disc (DVD), a magnetic disk, a semiconductor memory,and the like.

Note that, the program can be installed on the computer from theremovable recording medium 911 as described above, or can be downloadedto the computer via a communications network or a broadcast network andinstalled on the hard disk 905 incorporated. That is, for example, theprogram can be wirelessly transferred from a download site to thecomputer via an artificial satellite for digital satellite broadcasting,or can be transmitted to the computer via a network such as a Local AreaNetwork (LAN) or the Internet by wire.

The computer incorporates a Central Processing Unit (CPU) 902, and aninput/output interface 910 is connected to the CPU 902 via a bus 901.

The CPU 902 executes the program stored in the Read Only Memory (ROM)903 according to a command when the command is input by a user operatingan input unit 907 or the like via the input/output interface 910.Alternatively, the CPU 902 loads the program stored in the hard disk 905into a random access memory (RAM) 904 and executes the program.

The CPU 902 therefore performs the processing according to theabove-described flowchart or the processing performed by theconfiguration of the above-described block diagram. Then, the CPU 902causes the processing result to be output from an output unit 906 ortransmitted from a communication unit 908 via the input/output interface910 as necessary, and further, recorded on the hard disk 905, forexample.

Note that, the input unit 907 includes a keyboard, a mouse, amicrophone, and the like. Furthermore, the output unit 906 includes aLiquid Crystal Display (LCD), a speaker, and the like.

Here, in the present specification, the process performed by thecomputer in accordance with the program does not necessarily have to beperformed chronologically in the order described as the flowchart. Thatis, the process performed by the computer in accordance with the programalso includes processes executed in parallel or individually (forexample, parallel process or process by an object).

Furthermore, the program may be processed by one computer (processor) ormay be distributed and processed by a plurality of computers. Moreover,the program may be transferred to a remote computer and executed.

Moreover, in the present specification, a system means a set of aplurality of constituents (device, module (component), and the like),and it does not matter whether or not all of the constituents are in thesame cabinet. Thus, a plurality of devices that is accommodated in aseparate cabinet and connected to each other via a network and onedevice that accommodates a plurality of modules in one cabinet are bothsystems.

Note that, the embodiment of the present technology is not limited tothe embodiments described above, and various modifications are possiblewithout departing from the scope of the present technology.

For example, the present technology can adopt a configuration of cloudcomputing that shares one function in a plurality of devices via anetwork to process the function in cooperation.

Furthermore, each step described in the above flowchart can be executedby sharing in a plurality of devices, other than being executed by onedevice.

Moreover, in a case where a plurality of processes is included in onestep, the plurality of processes included in the one step can beexecuted by being shared in a plurality of devices, other than beingexecuted by one device.

Furthermore, the advantageous effects described in the presentspecification are merely examples and are not limited to them, and othereffects may be included.

REFERENCE SIGNS LIST

-   10 Image processing system-   11 Encoder-   21 A/D conversion unit-   22 Screen rearrangement buffer 22-   23 Calculation unit-   24 Orthogonal transform unit-   25 Quantization unit-   26 Lossless encoding unit-   27 Accumulation buffer-   28 Inverse quantization unit-   29 Inverse orthogonal transform unit-   30 Calculation unit-   31 a, 31 b Deblocking filter-   32 Frame memory-   33 Selection unit-   34 Intra prediction unit-   35 Motion prediction/compensation unit-   36 Predicted image selection unit-   37 Rate control unit-   41 Adaptive offset filter-   42 ALF-   51 Decoder-   61 Accumulation buffer-   62 Lossless decoding unit-   63 Inverse quantization unit-   64 Inverse orthogonal transform unit-   65 Calculation unit-   67 Screen rearrangement buffer-   68 D/A conversion unit-   69 Frame memory-   70 Selection unit-   71 Intra prediction unit-   72 Motion prediction/compensation unit-   73 Selection unit-   81 Adaptive offset filter-   82 ALF-   261 Boundary strength calculation unit-   310 Decision unit-   311 Filtering decision unit-   312 Filtering strength decision unit-   320 Filtering unit-   330 Line buffer-   340 Controller-   901 Bus-   902 CPU-   903 ROM-   904 RAM-   905 Hard disk-   906 Output unit-   907 Input unit-   908 Communication unit-   909 Drive-   910 Input/output interface-   911 Removable recording medium

1. An image processing device comprising: a decoding unit that decodes abitstream to generate a decoded image; a decision unit that decideswhether or not a deblocking filter is applied to color differencecomponents of pixels belonging to a line orthogonal to a block boundaryin two adjacent blocks adjacent to each other sandwiching the blockboundary of the decoded image generated by the decoding unit, by usingcolor difference components of pixels belonging to a line identical to aline used to decide whether or not the deblocking filter is applied toluminance components of the pixels belonging to the line orthogonal tothe block boundary; and a filtering unit that applies the deblocking tothe color difference components of the pixels for which it is decided bythe decision unit that the deblocking filter is applied.
 2. The imageprocessing device according to claim 1, wherein in a case where a colorformat of the decoded image is a YUV422 format, the decision unitperforms decision by using color difference components of pixelsbelonging to two horizontal lines located at both ends of a partialvertical block boundary in the adjacent blocks, among horizontal linesorthogonal to the partial vertical block boundary that is a unit ofprocessing when it is decided whether or not the deblocking filter isapplied to a vertical block boundary that is a block boundary in avertical direction.
 3. The image processing device according to claim 2,wherein a length of the partial vertical block boundary is four pixels,and the decision unit decides whether or not the deblocking filter isapplied to color difference components of pixels belonging to fourhorizontal lines of from first to fourth horizontal lines orthogonal tothe partial vertical block boundary, by using color differencecomponents of pixels belonging to the first and fourth horizontal lines.4. The image processing device according to claim 2, wherein thedecision unit decides whether or not the deblocking filter is applied,by using color difference components of pixels belonging to twohorizontal lines located at both ends of the partial vertical blockboundary in the adjacent blocks and two horizontal lines adjacent toeach other at a center of the partial vertical block boundary in theadjacent blocks, among horizontal lines orthogonal to the vertical blockboundary that is a block boundary in the vertical direction.
 5. Theimage processing device according to claim 4, wherein a length of thevertical block boundary is eight pixels, and the decision unit decideswhether or not the deblocking filter is applied to color differencecomponents of pixels belonging to eight horizontal lines of from firstto eighth horizontal lines orthogonal to the vertical block boundary, byusing color difference components of pixels belonging to the first,fourth, fifth, and eighth horizontal lines.
 6. The image processingdevice according to claim 1, wherein in a case where a color format ofthe decoded image is a YUV444 format, the decision unit decides whetheror not the deblocking filter is applied, by using color differencecomponents of pixels belonging to two horizontal lines located at bothends of a partial vertical block boundary in the adjacent blocks, amonghorizontal lines orthogonal to the partial vertical block boundary thatis a unit of processing when it is decided whether or not the deblockingfilter is applied to a vertical block boundary that is a block boundaryin a vertical direction.
 7. The image processing device according toclaim 6, wherein the decision unit decides whether or not the deblockingfilter is applied, by using color difference components of pixelsbelonging to two horizontal lines located at both ends of the partialvertical block boundary in the adjacent blocks and two horizontal linesadjacent to each other at a center of the partial vertical blockboundary in the adjacent blocks, among horizontal lines orthogonal tothe vertical block boundary that is a block boundary in the verticaldirection.
 8. The image processing device according to claim 1, whereinin a case where a color format of the decoded image is a YUV444 format,the decision unit decides whether or not the deblocking filter isapplied, by using color difference components of pixels belonging to twovertical lines located at both ends of a partial horizontal blockboundary in the adjacent blocks, among vertical lines orthogonal to thepartial horizontal block boundary that is a unit of processing when itis decided whether or not the deblocking filter is applied to ahorizontal block boundary that is a block boundary in a horizontaldirection.
 9. The image processing device according to claim 8, whereinthe decision unit decides whether or not the deblocking filter isapplied, by using color difference components of pixels belonging to twovertical lines located at both ends of the partial horizontal blockboundary in the adjacent blocks and two vertical lines adjacent to eachother at a center of the partial horizontal block boundary in theadjacent blocks, among vertical lines orthogonal to the horizontal blockboundary that is a block boundary in the horizontal direction.
 10. Animage processing method comprising: decoding a bitstream to generate adecoded image; deciding whether or not a deblocking filter is applied tocolor difference components of pixels belonging to a line orthogonal toa block boundary in two adjacent blocks adjacent to each othersandwiching the block boundary of the decoded image, by using colordifference components of pixels belonging to a line identical to a lineused to decide whether or not the deblocking filter is applied toluminance components of the pixels belonging to the line orthogonal tothe block boundary; and applying the deblocking to the color differencecomponents of the pixels for which it is decided that the deblockingfilter is applied.
 11. An image processing device comprising: a decisionunit that decides whether or not a deblocking filter is applied to colordifference components of pixels belonging to a line orthogonal to ablock boundary in adjacent blocks adjacent to each other sandwiching theblock boundary of a locally decoded image locally decoded when an imageis encoded, by using color difference components of pixels belonging toa line identical to a line used to decide whether or not the deblockingfilter is applied to luminance components of the pixels belonging to theline orthogonal to the block boundary; a filtering unit that applies thedeblocking to the color difference components of the pixels for which itis decided by the decision unit that the deblocking filter is applied,to generate a filter image; and an encoding unit that encodes the imageby using the filter image generated by the filtering unit.
 12. An imageprocessing method comprising: deciding whether or not a deblockingfilter is applied to color difference components of pixels belonging toa line orthogonal to a block boundary in adjacent blocks adjacent toeach other sandwiching the block boundary of a locally decoded imagelocally decoded when an image is encoded, by using color differencecomponents of pixels belonging to a line identical to a line used todecide whether or not the deblocking filter is applied to luminancecomponents of the pixels belonging to the line orthogonal to the blockboundary; applying the deblocking to the color difference components ofthe pixels for which it is decided that the deblocking filter isapplied, to generate a filter image; and encoding the image by using thefilter image.